# TeXForm > TeXForm parses, edits and normalizes LaTeX math using a knowledge base of command specifications. Open source, with packages for Python, JavaScript and Rust. TeXForm is open source (Apache-2.0). One Rust core backs packages for Python, JavaScript (WebAssembly, browser or Node.js) and Rust with the same API. - Install: `pip install texform`, `npm install texform`, or `cargo add texform`. The `texform` command-line tool ships with the Python package (`uv tool install texform`, or run once with `uvx texform`) and as the `texform-cli` crate (`cargo install texform-cli`). - Normalize with a `TransformEngine` and a profile. The four profiles form a ladder: `authoring` keeps the author's stylistic choices, `faithful` expands convenience macros while preserving rendering, `corpus` produces complete canonical forms for training labels, and `equiv` normalizes aggressively for equivalence checks and deduplication. - Start with Quick start, then Transforms for profiles and rules, and the API reference for your language. - Every page has a Markdown version: append `.md` to its URL (`/index.md` for the home page). `llms-full.txt` concatenates the documentation and API reference. # Documentation TeXForm parses, edits, and transforms LaTeX math from a command knowledge base, with one Rust core behind identical APIs in Rust, Python, and JavaScript. https://texform.dev/docs A formula parses into a real syntax tree, driven by xparse-style argument specifications for every command in a structured knowledge base; you get an editable `Document` tree to query and mutate, and a profile-based transform engine that normalizes formulas into canonical forms tuned for specific downstream uses. Programs that process LaTeX formulas — cleaning OCR training data, comparing formulas for equivalence, preparing pretraining corpora, building math editors — keep reinventing the same fragile layer of regular expressions and one-off heuristics that guess what `\frac`, `\over`, or `\quantity` mean. The hard part was never the string manipulation; it is knowing the argument structure of every command well enough to work on formulas as syntax trees instead of text. That is why TeXForm parses against a command knowledge base rather than matching patterns: a regex cannot tell that `\frac` takes two arguments while `\sqrt` takes an optional one followed by a mandatory one, because argument structure is not something the surface syntax reveals. [Parsing](https://texform.dev/docs/parsing) explains the full mental model. ## Three pillars [#three-pillars] TeXForm is built on three capabilities, each layered on the one before it: * **Knowledge-driven parser.** Formulas parse into real syntax trees, driven by argument specifications for every command in the built-in packages: `base`, `ams`, `physics`, `braket`, `bboldx`, `boldsymbol`, `color`, and `textmacros`. By default, a parser omits `physics` because it redefines `\bra`, `\ket`, and friends that `braket` also owns, so you turn it on explicitly when your formulas use physics-package notation. Unknown commands and unparseable fragments survive as explicit nodes under your control, never a crash. [Parsing](https://texform.dev/docs/parsing) covers the package set and how to change it. * **Editable document tree.** Parsing produces a `Document` — a DOM-style tree you can query, mutate, and serialize back to LaTeX. Edits are validated and return structured errors; no panic ever reaches the caller. The serializer guarantees text idempotency. * **Profile-based transform engine.** Normalization has no single correct answer, so TeXForm never imposes one true form. You pick a profile that names your downstream use, and a curated rule pipeline produces the canonical form for that case — from polished authoring output to aggressive corpus normalization. ## Where to start [#where-to-start] For corpus cleaning and formula comparison: 1. [Quick start](https://texform.dev/docs/quick-start) — normalize your first formula in Python, JavaScript, or Rust. 2. [Normalize a corpus](https://texform.dev/docs/normalize-a-corpus) — reuse an engine with error handling and statistics. For editors, linters, and converters: 1. [What TeXForm is](https://texform.dev/docs/what-is-texform) — understand the product boundary. 2. [Pipeline](https://texform.dev/docs/pipeline) — follow a formula through its output channels. 3. [Edit a document](https://texform.dev/docs/edit-a-document) — query and mutate the tree, then serialize it. --- # Installation Install TeXForm from pip, npm, or cargo, and verify it works in Python, JavaScript, or Rust. https://texform.dev/docs/installation TeXForm ships the same Rust core through three package registries. Install it for your language, then run the verification snippet to confirm the native module loaded. **Python** ```bash pip install texform ``` The package is a compiled extension distributed as prebuilt wheels and requires Python 3.10 or newer. Bundled `.pyi` type stubs and a `py.typed` marker give you full type checking and editor completion with no separate stubs package. ```python import texform engine = texform.TransformEngine(profile="corpus") print(engine.normalize(r"a \over b")) ``` ```text \frac { a } { b } ``` **JavaScript** ```bash npm install texform ``` The package is powered by WebAssembly, with TypeScript declarations bundled — no separate `@types` package. It ships two entry points for loading the `.wasm` module, and choosing the right one is the most common pitfall with a WASM package: `texform/node` targets Node.js, and `texform/bundler` targets browser-oriented bundlers such as Vite or webpack 5. The bare `texform` import resolves to the Node entry under Node.js and the bundler entry under a bundler, so most projects can import `texform` directly; force a specific entry only when the automatic resolution picks the wrong one. The bundler entry initializes the WebAssembly module at module load time, which needs a bundler that supports top-level `await` and `.wasm` assets. [JavaScript and WebAssembly](https://texform.dev/docs/javascript-and-wasm) walks through both entry points, the bundler configuration they expect, and the handle lifecycle. ```js import { TransformEngine } from 'texform/node'; const engine = new TransformEngine({ profile: 'corpus' }); console.log(engine.normalize('a \\over b')); ``` ```text \frac { a } { b } ``` **Rust** ```bash cargo add texform ``` The `texform` facade crate is the only public, stability-guaranteed entry point. It depends on internal `texform-*` crates that are published only because crates.io requires it; their APIs may change in any release, so do not depend on them directly. ```rust use texform::{Profile, TransformEngine}; let engine = TransformEngine::builder().profile(Profile::Corpus).build()?; let result = engine.normalize(r"a \over b")?; assert_eq!(result, r"\frac { a } { b }"); ``` Once the verification snippet runs, continue to the [Quick start](https://texform.dev/docs/quick-start) for the full normalize-and-inspect path. ## Command-line tool [#command-line-tool] The `texform` command serves shell scripts and tools that call TeXForm across a process boundary. It ships with the Python package; install it as a standalone tool, or run it once with `uvx texform`: ```bash uv tool install texform # or: pipx install texform texform normalize --profile corpus 'a \over b' ``` ```text \frac { a } { b } ``` With a Rust toolchain, `cargo install texform-cli` builds the same tool as a native binary. [Command line](https://texform.dev/docs/command-line) covers every command, the JSON output, and the `texform serve` protocol. ## Next steps [#next-steps] - [Quick start](https://texform.dev/docs/quick-start) — Normalize a formula, switch profiles, and open the document tree underneath. - [JavaScript and WebAssembly](https://texform.dev/docs/javascript-and-wasm) — Entry points, bundler setup, and the handle lifecycle for the npm package. - [Playground](https://texform.dev/docs/playground) — Try profiles and diagnostics in the browser before you install anything. --- # Quick start Normalize your first formula, pick a profile, and look at the document tree underneath — in 10 minutes. https://texform.dev/docs/quick-start This walks the core TeXForm path in about ten minutes: normalize a formula, choose the profile that controls how aggressive normalization is, and look at the document tree that sits underneath. Every step shows the output and one line on what just happened. ## Install [#install] Install TeXForm for your language. See [Installation](https://texform.dev/docs/installation) for supported versions and verification. **Python** ```bash pip install texform ``` **JavaScript** ```bash npm install texform ``` **Rust** ```bash cargo add texform ``` ## Normalize your first formula [#normalize-your-first-formula] Build a `TransformEngine` with the `corpus` profile and normalize a `physics`-package derivative. `physics` is not in the default package set, so the engine gets a knowledge base that names the packages it loads. The convenience macro `\dv{f}{x}` expands into the universal `\frac` form that renders on any plain MathJax or KaTeX deployment. **Python** ```python import texform kb = texform.KnowledgeBase(packages=["base", "ams", "physics"]) engine = texform.TransformEngine(profile="corpus", knowledge_base=kb) print(engine.normalize(r"\dv{f}{x}")) ``` ```text \frac { \mathrm { d } f } { \mathrm { d } x } ``` **JavaScript** ```js import { KnowledgeBase, TransformEngine } from 'texform'; const knowledgeBase = new KnowledgeBase({ packages: ['base', 'ams', 'physics'] }); const engine = new TransformEngine({ profile: 'corpus', knowledgeBase }); console.log(engine.normalize('\\dv{f}{x}')); ``` ```text \frac { \mathrm { d } f } { \mathrm { d } x } ``` **Rust** ```rust use texform::{KnowledgeBase, Profile, TransformEngine}; let kb = KnowledgeBase::builder().packages(&["base", "ams", "physics"]).build()?; let engine = TransformEngine::builder() .knowledge_base(kb) .profile(Profile::Corpus) .build()?; let result = engine.normalize(r"\dv{f}{x}")?; println!("{}", result); ``` ```text \frac { \mathrm { d } f } { \mathrm { d } x } ``` What just happened: the engine parsed `\dv{f}{x}`, recognized it as a `physics` derivative macro, and rewrote it into a form that no longer depends on the `physics` package. The default serialized style is spaced, which is why braces and operators are separated. ## Pick a profile [#pick-a-profile] The profile is the core decision: it names your downstream use and selects how far normalization goes. Run the same input through the `authoring` profile and compare. **Python** ```python import texform kb = texform.KnowledgeBase(packages=["base", "ams", "physics"]) corpus = texform.TransformEngine(profile="corpus", knowledge_base=kb) authoring = texform.TransformEngine(profile="authoring", knowledge_base=kb) print(corpus.normalize(r"\dv{f}{x}")) print(authoring.normalize(r"\dv{f}{x}")) ``` ```text \frac { \mathrm { d } f } { \mathrm { d } x } \dv { f } { x } ``` **JavaScript** ```js import { KnowledgeBase, TransformEngine } from 'texform'; const knowledgeBase = new KnowledgeBase({ packages: ['base', 'ams', 'physics'] }); const corpus = new TransformEngine({ profile: 'corpus', knowledgeBase }); const authoring = new TransformEngine({ profile: 'authoring', knowledgeBase }); console.log(corpus.normalize('\\dv{f}{x}')); console.log(authoring.normalize('\\dv{f}{x}')); ``` ```text \frac { \mathrm { d } f } { \mathrm { d } x } \dv { f } { x } ``` **Rust** ```rust use texform::{KnowledgeBase, Profile, TransformEngine}; let kb = KnowledgeBase::builder().packages(&["base", "ams", "physics"]).build()?; let corpus = TransformEngine::builder().knowledge_base(kb.clone()).profile(Profile::Corpus).build()?; let authoring = TransformEngine::builder().knowledge_base(kb).profile(Profile::Authoring).build()?; println!("{}", corpus.normalize(r"\dv{f}{x}")?); println!("{}", authoring.normalize(r"\dv{f}{x}")?); ``` ```text \frac { \mathrm { d } f } { \mathrm { d } x } \dv { f } { x } ``` What just happened: `corpus` expanded the macro because a training corpus should not depend on the `physics` package, while `authoring` kept the author's shorthand. Same input, different target, different output — no profile is "better", only fitter for a downstream use. The four profiles and the rules behind them are covered in [Transforms](https://texform.dev/docs/transforms). ## Look under the hood [#look-under-the-hood] Beneath `normalize` there is a tree you can use any time. Parse a formula directly, check that the parse is clean, and serialize it back to LaTeX. **Python** ```python import texform result = texform.Parser().parse(r"\frac{x}{y}") document = result["document"] print(document.has_errors()) print(document.to_latex()) ``` ```text False \frac { x } { y } ``` **JavaScript** ```js import { Parser } from 'texform'; const result = new Parser().parse('\\frac{x}{y}'); const document = result.document; console.log(document.hasErrors()); console.log(document.toLatex()); ``` ```text false \frac { x } { y } ``` **Rust** ```rust use texform::Parser; let parser = Parser::builder().build()?; let parsed = parser.parse(r"\frac{x}{y}"); if let Some(document) = parsed.document() { println!("{}", document.has_errors()); println!("{}", document.to_latex()?); } ``` ```text false \frac { x } { y } ``` What just happened: `parse` returned a `Document` — the editable working tree. `has_errors()` confirms the parse is complete (a tree with holes would be read-only and could not be normalized), and `to_latex()` serializes it back. Normalization is this same tree with a rule pipeline applied; you can also walk and edit it directly. - [Try in Playground](https://play.texform.dev) — Run this flow in the browser, no install needed. - [Concepts: Transforms](https://texform.dev/docs/transforms) — The four profiles and the multi-phase pipeline behind normalization. - [Guide: Normalize a corpus](https://texform.dev/docs/normalize-a-corpus) — Run one reused engine over a whole dataset. --- # Playground Try TeXForm in the browser: compare profiles, read parse diagnostics, and inspect the tree without installing anything. https://texform.dev/docs/playground The playground at [play.texform.dev](https://play.texform.dev) runs the real TeXForm WebAssembly build in your browser. You type a formula, and the same parser, transform engine, and serializer you would get from `pip install texform` or `npm install texform` produce the output — nothing is simulated, and nothing is sent to a server. It is the fastest way to answer the questions that come up before you write any code: what does this profile actually do to my formula, why did this formula fail to parse, and what does the tree look like underneath. ## What to try there [#what-to-try-there] * **Compare profiles on your own formula.** Paste a formula and switch between `authoring`, `faithful`, `corpus`, and `equiv`. Profile choice is the one decision that changes normalization output the most, and watching all four results on input you care about is more convincing than any table. See [Transforms](https://texform.dev/docs/transforms) for what each profile targets. * **Read parse diagnostics against the source.** Diagnostics are listed with the source span they point at, so a malformed formula shows you which fragment the parser could not read instead of a single opaque failure. The concepts behind the three parse outcomes are in [Parsing](https://texform.dev/docs/parsing), and the programmatic version is in [Handle parse errors](https://texform.dev/docs/handle-parse-errors). * **Inspect the document tree.** The tree view shows the parsed nodes, their argument slots, and the argspec the parser used for each command — useful when a command binds its arguments differently from what you expected. * **Check which packages are loaded.** The package list shows the knowledge packages in effect, which is the quickest way to see why `\ket` behaves the way it does with and without `braket`. * **Try serialization options.** The serialization settings map to the same options you would pass to `to_latex` / `toLatex`, so you can preview a spacing or script-order choice before committing to it in code. See [Configure output style](https://texform.dev/docs/configure-output-style). * **See the output rendered.** The playground renders the normalized result, which is a direct check on the point of the `corpus` profile: output that needs no extra packages to display. ## The version it runs [#the-version-it-runs] > **The playground may lag behind the latest release** > > The playground is a separate application that depends on the published npm `texform` package, so it tracks published releases rather than unreleased work. When an output there disagrees with these docs, the installed package is the authority. ## What it does not do [#what-it-does-not-do] There is no URL parameter for preloading a formula. Query strings are ignored, so a link to the playground always opens the default state; share the formula itself rather than a prefilled link. The playground also has no API and no persistence beyond your own browser, and it is not a substitute for the corpus workflows in [Normalize a corpus](https://texform.dev/docs/normalize-a-corpus) — a browser tab is the wrong place to push a million formulas through. ## Next steps [#next-steps] - [Installation](https://texform.dev/docs/installation) — Install the package for Python, JavaScript, or Rust and verify it loads. - [Quick start](https://texform.dev/docs/quick-start) — Reproduce the playground flow in code in about ten minutes. - [Transforms](https://texform.dev/docs/transforms) — What each profile targets and which rules it selects. --- # JavaScript and WebAssembly Pick the right texform entry point for Node or a bundler, configure top-level await and .wasm assets, and manage handle lifetimes. https://texform.dev/docs/javascript-and-wasm The npm `texform` package is the Rust core compiled to WebAssembly, wrapped in a small JavaScript layer that adds structured errors and camelCase names. Everything in [Quick start](https://texform.dev/docs/quick-start) works unchanged in JavaScript; this page covers the two things that are specific to a WASM package — getting the module loaded, and knowing when a handle is released. ## Three entry points [#three-entry-points] The package declares three specifiers, and they resolve to two different loaders: | Import | Resolves to | Use it when | | ----------------- | ----------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- | | `texform` | The Node loader under Node.js, the bundler loader under a bundler | Almost always | | `texform/node` | The Node loader, unconditionally | Conditional resolution picks the wrong one, or a tool reads the package without Node conditions | | `texform/bundler` | The bundler loader, unconditionally | You bundle for the browser but the tool reports a Node environment | The bare `texform` specifier is the one to reach for. It maps to `./node/index.js` (or `./node/index.cjs` for `require`) under Node's conditions, and to `./bundler/index.js` for `browser` and default conditions. The two subpath specifiers exist to override that decision, not to be the normal choice — all three ship the same TypeScript declarations and the same API. **Automatic** ```js import { TransformEngine } from 'texform'; const engine = new TransformEngine({ profile: 'corpus' }); console.log(engine.normalize('a \\over b')); ``` **Node** ```js import { TransformEngine } from 'texform/node'; const engine = new TransformEngine({ profile: 'corpus' }); console.log(engine.normalize('a \\over b')); ``` The Node loader requires the `.wasm` module through `createRequire`, so it is fully synchronous: by the time the `import` statement resolves, the engine is ready. CommonJS works too — `require('texform')` resolves to the CommonJS build of the same wrapper. **Bundler** ```js import { TransformEngine } from 'texform/bundler'; const engine = new TransformEngine({ profile: 'corpus' }); console.log(engine.normalize('a \\over b')); ``` The bundler loader awaits WebAssembly instantiation at module scope, so importing it is enough — there is no `init()` for you to call and no readiness flag to check. ## What the bundler entry needs [#what-the-bundler-entry-needs] The bundler loader resolves the `.wasm` file as an asset URL and then awaits its instantiation at the top level of the module. That puts two requirements on your build: * **Top-level `await` must survive bundling.** The loader is an ES module with a top-level `await`, which means the output format has to be ESM and the target has to be modern. Vite and webpack 5 handle this; a build configured to emit CommonJS or to target older browsers will fail here, usually with an error naming top-level `await` rather than TeXForm. * **`.wasm` must be emitted as an asset.** The module is fetched at runtime from a URL, not inlined into the JavaScript bundle. A bundler that does not know how to emit `.wasm` will either drop the file or try to parse it as JavaScript. Because the import is awaited, the first module in your graph that imports `texform` becomes async. That is normally invisible, but it does mean a synchronous entry point cannot import TeXForm at the top level — move the import into an async boundary, such as a dynamic `import()` inside the handler that needs it. ## Handle lifetimes [#handle-lifetimes] `Parser`, `TransformEngine`, `Document`, and `Node` are handles to objects that live in WebAssembly memory, not plain JavaScript values. Each one exposes `free()` and `[Symbol.dispose]()` to release the underlying object. ```js import { TransformEngine } from 'texform'; const formulas = ['a \\over b', '{\\bf x}']; const engine = new TransformEngine({ profile: 'corpus' }); try { for (const formula of formulas) { console.log(engine.normalize(formula)); } } finally { engine.free(); } ``` With explicit resource management (TypeScript 5.2 or newer, or a runtime that supports `using`), the same thing reads as a declaration: ```ts import { TransformEngine } from 'texform'; { using engine = new TransformEngine({ profile: 'corpus' }); console.log(engine.normalize('a \\over b')); } ``` Two habits keep this simple. Build one engine and reuse it across a whole batch rather than constructing one per formula, as in [Normalize a corpus](https://texform.dev/docs/normalize-a-corpus) — that is the right call for performance anyway. And release a `Document` only after you are done with the `Node` handles you obtained from it, since those nodes are references back into that document rather than independent copies. ## Configuration is overlays, not classes [#configuration-is-overlays-not-classes] The JavaScript API has no configuration classes. Every per-call config is a plain object overlay: you pass only the keys you want to change, and omitted keys (or `null` / `undefined`) keep the value the profile or parser already has. ```js import { TransformEngine } from 'texform'; const engine = new TransformEngine({ profile: 'corpus' }); engine.normalize('a \\over b', { rewrite: { enabled: false } }); engine.normalize('a \\over b', { rejectUnknown: true, flattenGroups: { enabled: false } }); ``` Overlays are validated strictly. An unknown key, a snake_case key, an array where an object belongs, or a wrong scalar type throws `TexformConfigError` with the camelCase path of the offending field. Enum *values* stay snake_case even though keys are camelCase — `{ scriptOrder: 'sub_first' }` is correct. When you need to know what is actually in force rather than what you overrode, ask: `parser.defaultParseConfig()`, `engine.defaultParseConfig()`, and `engine.defaultTransformConfig()` return the complete resolved configuration. ## Errors [#errors] Every failure that crosses the WebAssembly boundary arrives as a structured JavaScript exception; no Rust panic reaches your code. The classes are `TexformError` and its subclasses `TexformParseError`, `TexformEditError`, `TexformConfigError`, and `TexformTransformError`. A `TexformParseError` additionally carries `diagnostics` and, when one exists, the partial `document`. ## Next steps [#next-steps] - [Quick start](https://texform.dev/docs/quick-start) — The core normalize-and-inspect path, with a JavaScript tab at every step. - [Troubleshooting](https://texform.dev/docs/troubleshooting) — What to check when the module will not load or a config is rejected. - [Serialization](https://texform.dev/docs/serialization) — The output channels and the options that shape the LaTeX string. --- # Command line Install the texform command to normalize, parse, and tokenize formulas from the shell, look up knowledge-base records, and serve the normalizer protocol. https://texform.dev/docs/command-line The `texform` command exposes the same engine as the Rust, Python, and JavaScript libraries to shell scripts and to tools that call TeXForm across a process boundary. It reads formulas from an argument or from stdin, and prints either plain text or one JSON value per formula. ## Installation [#installation] The `texform` command ships with the Python package, which has prebuilt wheels for Linux, macOS, and Windows. Install it as a standalone tool: ```bash uv tool install texform # or: pipx install texform ``` To run it once without installing, use `uvx texform`. In an environment that already uses the Python library, `pip install texform` installs the command too, also available as `python -m texform`. With a Rust toolchain, `cargo install texform-cli` builds the same tool from source as a native binary. Both behave identically; the Python install adds about 10 ms of interpreter startup to each run, which matters only when calling the command once per formula. For many formulas, use [`--lines`](#formula-input) or [`texform serve`](#serve-the-normalizer-protocol) instead. Check that it runs: ```bash texform --version ``` ```text texform ``` When the tool is built from a git checkout, the version line also carries the abbreviated commit hash and its date, as in `texform ( )`, followed by `, dirty` if the build inputs differed from that commit. A build installed from PyPI or crates.io has no git information, so only the version appears. The [serve protocol](#serve-the-normalizer-protocol) reports the same identity in `serverInfo`. ## Usage [#usage] ```text texform [--packages ] [OPTIONS] ``` | Command | Purpose | | ------------------ | -------------------------------------------------------------------- | | `normalize` | Normalize formulas with a transform profile | | `parse` | Parse formulas and inspect their syntax trees | | `tokenize` | Split the spaced serialization of formulas into typed tokens | | `info` | Show knowledge-base records for a control sequence or an environment | | `packages` | List the built-in knowledge packages | | `argspec validate` | Check an xparse-style argument specification | | `serve` | Serve the normalizer protocol to another process | `texform --help` lists every option of a command. The one global option is `--packages `, which picks the knowledge packages to load. It defaults to the same seven packages as the libraries (`base`, `ams`, `braket`, `textmacros`, `bboldx`, `boldsymbol`, and `color`); `physics` is opt-in, for the reasons given in [Parsing](https://texform.dev/docs/parsing). An unknown name is a usage error with exit status `2`. The `packages` and `argspec validate` commands do not use it. ## Formula input [#formula-input] `normalize`, `parse`, and `tokenize` read formulas from one of three sources: * **The `LATEX` argument** is one formula. Quote it with single quotes, such as `'\sqrt{a+b}'`, so the shell preserves backslashes. Put `--` before a formula that starts with `-`, for example `texform parse -- '-x^2'`; unknown options are rejected rather than read as formulas. * **Stdin without an argument** is one formula: everything on stdin, minus one trailing newline (`\n` or `\r\n`). A formula with line breaks, such as an environment, stays whole. * **`--lines`** makes every stdin line (ending in `\n` or `\r\n`) its own formula, and an empty line an empty formula. Lines are processed as they arrive, so the command can sit at the end of a pipe. Input must be UTF-8. ## Output and failures [#output-and-failures] `normalize` prints one line of LaTeX per formula, and `tokenize` prints one line containing a token list. `parse` prints a syntax tree; with `--lines`, the trees are separated into blocks headed by ``. Add `--json` (with `--lines` for batches) to get one JSON value per input line instead. A failing formula does not stop the run, and the rest of the input is still processed: * In text mode, the error and diagnostics go to stderr. `parse` also shows any partial tree on stdout. Under `--lines`, a failed normalization or tokenization writes an empty placeholder line, so output lines still match input lines. * In JSON mode, every formula produces one JSON value: successful normalization and parsing produce objects, successful tokenization produces an array, and any failure produces `{"ok":false,"error":{...}}`. Processing errors, including recovered panics, do not write to stderr; usage and configuration errors still do. * When the downstream reader closes stdout (for example `| head -n 5`), processing stops immediately and the command exits successfully without reading the remaining stdin. Diagnostics are colored only when stderr is a terminal and `NO_COLOR` is unset or empty. ### Exit status [#exit-status] | Status | Meaning | | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `0` | Every formula succeeded, or the lookup or validation succeeded | | `1` | At least one formula failed, `info` found nothing, or `argspec validate` rejected the specification | | `2` | Usage, configuration, or I/O error, for example an unknown option or package, a missing `--profile`, an invalid `--config`, or unreadable or non-UTF-8 input. Output may be incomplete. | ### Configuration with `--config` [#configuration-with---config] `--config` takes a JSON object, inline or as `@FILE` to read it from a file. Its values are layered over the defaults, and nested objects may be partial. Unknown keys and a zero `rewrite.max_iterations` are rejected before any formula is read, with exit status `2`, and the error names the offending path, such as `invalid --config: rewrite.enabld: unknown field ...`. | Command | Config shape | Keys | | --------------------------------- | ------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------- | | `normalize`, `tokenize --profile` | Normalize config over the profile defaults | `reject_unknown`, `abort_on_error`, `max_group_depth`, `lower_attributes`, `rewrite`, `finalize_ast`, `flatten_groups` | | `parse`, `tokenize` | Parse config over the parser defaults | `reject_unknown`, `abort_on_error`, `max_group_depth` | The normalize config has the same keys and resolves the same way as the keyword arguments of the Python binding, the camelCase options of the JavaScript binding, and the `overrides` object of the serve protocol's `configure` request. [Transforms](https://texform.dev/docs/transforms) explains what each phase does. ## texform normalize [#texform-normalize] ```bash texform normalize --profile corpus 'a \over b' texform normalize --profile corpus --style compact 'a \over b' texform normalize --profile authoring --config '{"rewrite":{"enabled":false}}' --lines < formulas.txt texform normalize --profile corpus --report --json 'a \over b' ``` `--profile` is required and takes one of `authoring`, `faithful`, `corpus`, or `equiv`; the [profiles](https://texform.dev/docs/transforms#pick-a-profile) are the same ones the libraries use. The output is identical to what a `TransformEngine` returns for the same packages, profile, and config. `--report` adds the transform report to each result and requires `--json`. `--style` selects the output style, `spaced` (the default) or `compact`; it is a separate flag rather than a `--config` key. See [Configure output style](https://texform.dev/docs/configure-output-style). The example below feeds three formulas, one of them truncated. The failure leaves an empty line in place, `2>/dev/null` hides its diagnostics, and the exit status is `1` because one formula failed. `\dv` is a `physics` command, so without that package it is an unknown command and passes through unchanged: ```console $ printf '%s\n' 'a \over b' '\frac{a' '\dv{f}{x}' | texform normalize --profile corpus --lines 2>/dev/null \frac { a } { b } \dv { f } { x } $ echo $? 1 ``` Loading `physics` through `--packages` turns the same line into the universal `\frac` form: ```console $ texform --packages base,ams,physics normalize --profile corpus '\dv{f}{x}' \frac { \mathrm { d } f } { \mathrm { d } x } ``` ## texform parse [#texform-parse] Parses formulas without normalizing them. The default output is a compact syntax tree. `--verbose` prints the detailed tree as pretty-printed JSON and conflicts with `--json`, which emits a compact result object for scripts. Success means a complete document: a partial document containing error nodes still exits with status `1`, but its tree remains visible. | Parse result | Text mode (including `--verbose`) | `--json` | | ----------------- | ----------------------------------------------------------------- | ------------------------------------------------- | | Complete document | Syntax tree on stdout; diagnostics, if any, as warnings on stderr | `ok: true` with `syntax` and `diagnostics` | | Partial document | Partial tree on stdout; error and diagnostics on stderr | `ok: false` with `error` and the partial `syntax` | | No document | Error and diagnostics on stderr | `ok: false` with `error` | ```console $ texform parse '\sqrt{a+b}' Root(Math) [ Command(\sqrt, known=true) [ Arg(None) Arg(Mandatory, no_leading_space=false): Group(Math, Implicit) [ Chars("a+b") ] ] ] ``` The diagnostic source is named ``, ``, or `` after where the formula came from. Under `--lines`, each tree is headed by `` and followed by a blank line, including empty formulas and failures. The tree is the same one [Parsing](https://texform.dev/docs/parsing) describes. ## texform tokenize [#texform-tokenize] Lists tokens from the spaced serialization of each formula, not from the original input; `tokenize` has no `--style` flag. With `--profile`, the formula is normalized before tokenization. Partial documents fail as in `parse`. ```console $ texform tokenize '\text{a b}' [control_seq("\\text"), delim("{"), text("a b", mode=text), delim("}")] ``` The default output is a compact list of `kind("text")` entries. Human-readable kinds abbreviate `character` to `char`, `control_sequence` to `control_seq`, and `delimiter` to `delim`; `text`, `raw`, and `error` keep their names, and JSON keeps the full kind names. Math mode is implicit, so only non-math tokens show their mode. Text uses JSON string quoting to preserve spaces and escape newlines, quotes, and backslashes. An empty formula prints `[]`. Each formula occupies one output line. `--json` emits an array of token objects directly, with no `ok` or `latex` wrapper. Each object has `text`, `kind`, `mode`, `start_byte`, and `end_byte`. Under `--lines`, each successful input line produces one array on one output line, and a failure produces the standard error object, so output lines keep matching input lines. Offsets refer to the spaced serialization (the normalized serialization when `--profile` is given), not to the input string. Spaces between tokens belong to that serialization, so joining token texts with spaces is not a way to reconstruct it, particularly in text mode. ## texform info [#texform-info] ```bash texform info '\frac' texform info --env align texform info --mode text '\textbf' ``` Looks up a name in the knowledge base of the loaded packages, in math mode unless `--mode text` is given. * A control sequence such as `\frac` shows its command record and, when one exists, its character record. Character commands such as `\alpha` have both: the character record gives the Unicode value, and the command record shows that the name parses as a command without arguments. When both records are shown, a note identifies the command record as the one used for parsing. * `--env NAME` shows an environment record. * Anything that is neither a control sequence nor used with `--env` is a usage error. Characters are looked up by their control-sequence name; there is no lookup by Unicode character. When nothing is found, the exit status is `1`. ```console $ texform info '\alpha' command: \alpha kind: prefix mode: math packages: base argspec: (no arguments) character: \alpha unicode: α mode: math variant: italic package: base Parsing uses the command record above. The character record is separate metadata. ``` ## texform packages [#texform-packages] Lists the built-in knowledge packages with the number of commands and environments in each. The catalog includes `physics`, even though it is not loaded by default. ```console $ texform packages ams 35 commands, 28 environments base 270 commands, 7 environments bboldx 12 commands, 0 environments boldsymbol 1 command, 0 environments braket 11 commands, 0 environments color 5 commands, 0 environments physics 182 commands, 1 environment textmacros 78 commands, 0 environments ``` ## texform argspec validate [#texform-argspec-validate] Checks an xparse-style argument specification and describes each argument slot. An invalid specification is reported on stderr with exit status `1`. The syntax is described in [ArgSpec](https://texform.dev/docs/argspec). ```console $ texform argspec validate 'm O{default} m' valid: 3 arguments 1. required, math content 2. optional, math content 3. required, math content ``` ## JSON output [#json-output] `--json` output uses the same data structures as the Python and JavaScript bindings, so field names and values match the Python binding exactly and the JavaScript binding up to camelCase. Member order is not significant. | Command | Output line | | ------------------------------- | ------------------------------------------------------------------------------------------------ | | `normalize` | `{"ok":true,"output":string,"report"?:object}` | | `parse` | `{"ok":true,"syntax":object,"diagnostics":[object]}` | | `tokenize` | `[{"text","start_byte","end_byte","kind","mode"}]` | | Any formula command, on failure | `{"ok":false,"error":{"kind","message","diagnostics"},"syntax"?:object}` | | `info` | `{"command"?:object,"character"?:object,"environment"?:object}`, or `null` when nothing is found | | `packages` | `[{"name","commands","environments"}]` on one line | | `argspec validate` | `{"valid","error","arg_count","parsed"}` | * `error.kind` is `parse` (the formula did not parse into a complete tree), `transform` (a normalization step failed), or `internal` (an internal error, including a panic inside TeXForm, which fails only that formula). * `error.diagnostics` lists parse diagnostics for `parse` failures and is empty otherwise. Each diagnostic has `kind`, `message`, `span` (`start` and `end` byte offsets into the formula), `expected`, `found`, and `contexts`. Failures carry `syntax` only from `parse`, and only when a partial document exists. * `syntax` is the serialized syntax tree that `Document.to_syntax()` returns in Python and `toSyntax()` in JavaScript. * `report` is the transform report. Its fields are diagnostic and carry no compatibility promise. * Each token has `kind` of `control_sequence`, `character`, `delimiter`, `text`, `raw`, or `error`, and `mode` of `math` or `text`. The offsets are UTF-8 byte offsets, like the [spans](https://texform.dev/docs/handle-parse-errors) the libraries report. * The `info` records are the same results the bindings return from their command, character, and environment lookups. ```console $ texform normalize --profile corpus --json '\frac{a' {"ok":false,"error":{"kind":"parse","message":"parse produced an incomplete document","diagnostics":[{"kind":"argument-validation","message":"unclosed brace argument","span":{"start":5,"end":7},"expected":[],"found":null,"contexts":[{"label":"command argument","span":{"start":5,"end":7}}]}]}} $ texform tokenize --json 'x^2' [{"text":"x","start_byte":0,"end_byte":1,"kind":"character","mode":"math"},{"text":"^","start_byte":2,"end_byte":3,"kind":"character","mode":"math"},{"text":"{","start_byte":4,"end_byte":5,"kind":"delimiter","mode":"math"},{"text":"2","start_byte":6,"end_byte":7,"kind":"character","mode":"math"},{"text":"}","start_byte":8,"end_byte":9,"kind":"delimiter","mode":"math"}] ``` ## Serve the normalizer protocol [#serve-the-normalizer-protocol] `texform serve` is a long-running process that normalizes formulas on request. It speaks JSON-RPC 2.0 over stdin and stdout, so another program can start it once and send many formulas without paying process start-up and rule-plan compilation for each one. This is the command-line counterpart of [reusing one engine](https://texform.dev/docs/normalize-a-corpus). The protocol is independent of TeXForm: any normalizer can implement it, and a client written against it can measure any implementation. ```bash texform serve [--packages ] ``` `--packages` only sets the default for `configure` requests that omit `packages`. ### Transport [#transport] * Each message is one line of UTF-8 JSON terminated by `\n`. Messages must not contain raw newlines; JSON string escapes such as `\n` are fine. * The server reads requests from stdin and writes only responses to stdout, one per line, flushing after each. stderr carries human-readable logs that clients must not parse. * Requests are processed one at a time in arrival order, and responses are written in the same order. Clients may pipeline requests or wait for each response. * A request without an `id` member is a notification: it is executed, but no response is written, even on failure. `id` may be a string, a number, or `null`. * Batch requests (JSON arrays) are not supported and receive a single `-32600` error. Blank lines are ignored. * When stdin reaches end of file, the server finishes the requests it has read and exits with status `0`, whether or not `shutdown` was sent. A session looks like this, where `→` is client to server and `←` is server to client, with some lines abbreviated: ```text → {"jsonrpc":"2.0","id":0,"method":"initialize","params":{}} ← {"jsonrpc":"2.0","id":0,"result":{"protocolVersion":1,"serverInfo":{"name":"texform","version":"...","commit":"...","commitDate":"...","dirty":false}}} → {"jsonrpc":"2.0","id":1,"method":"configure","params":{"id":"authoring-no-rewrite","config":{"profile":"authoring","overrides":{"rewrite":{"enabled":false}}}}} ← {"jsonrpc":"2.0","id":1,"result":{"resolved":{"profile":"authoring","packages":["ams","base","..."],"rewrite":{"enabled":false,"max_iterations":100},"...":"..."}}} → {"jsonrpc":"2.0","id":2,"method":"normalize","params":{"config":"authoring-no-rewrite","latex":"\\frac12","timing":true}} ← {"jsonrpc":"2.0","id":2,"result":{"output":"\\frac { 1 } { 2 }","timing":{"parse_ns":41250,"transform_ns":35980,"serialize_ns":2410}}} → {"jsonrpc":"2.0","id":3,"method":"normalize","params":{"config":"authoring-no-rewrite","latex":"\\frac{a"}} ← {"jsonrpc":"2.0","id":3,"error":{"code":1,"message":"parse produced an incomplete document","data":{"kind":"parse","diagnostics":[...]}}} → {"jsonrpc":"2.0","id":4,"method":"shutdown","params":{}} ← {"jsonrpc":"2.0","id":4,"result":null} ``` ### Methods [#methods] | Method | `params` | `result` | | ------------ | ---------------------------------------------------------------------------------------------------- | ----------------------------------------------------- | | `initialize` | `{}` | `{"protocolVersion":1,"serverInfo":{...}}` | | `configure` | `{"id":string,"config":{"profile":string,"style"?:string,"packages"?:[string],"overrides"?:object}}` | `{"resolved":object}` | | `normalize` | `{"config":string,"latex":string,"timing"?:bool,"report"?:bool}` | `{"output":string,"timing"?:object,"report"?:object}` | | `shutdown` | `{}` | `null` | `params` must be an object; `initialize` and `shutdown` also accept omitted `params`. Unknown members of `params` itself are ignored, while unknown members of `config` and `overrides` are rejected (see [Versioning](#versioning)). #### initialize [#initialize] Must be the first request. Any other method received earlier fails with `-32002`. Repeating `initialize` is harmless and returns the same result. `serverInfo` identifies the build, like `texform --version`: | Field | Type | Meaning | | ------------ | ---------------- | -------------------------------------------------------------------- | | `name` | string | `"texform"` | | `version` | string | Crate version | | `commit` | string or `null` | Full commit hash, or `null` without git information | | `commitDate` | string or `null` | Committer date of `commit` (`YYYY-MM-DD`) | | `dirty` | bool | Build inputs differed from `commit`; `false` without git information | #### configure [#configure] Builds a normalizer from `config` and stores it under `id` for later `normalize` requests. Configuring an existing `id` replaces its entry; a failed `configure` leaves the previous entry unchanged. * `profile` is one of `authoring`, `faithful`, `corpus`, or `equiv`. It selects the rule set and the default config. * `style` is the output style of `normalize` results, `spaced` (the default) or `compact`. It sits beside `profile`, not inside `overrides`. * `packages` lists the knowledge packages to load. When omitted, the server's `--packages` value applies, and without that, the same seven default packages as the libraries (`physics` is opt-in). Clients that need a fixed selection should specify it explicitly and record `resolved`. * `overrides` are config values layered over the profile defaults. They accept any subset of the keys in `resolved` other than `style`, `profile`, and `packages`, with the same nesting; nested objects may also be partial. `resolved` is the complete effective config, suitable for recording alongside results: ```json { "style": "spaced", "profile": "authoring", "packages": ["ams", "base", "bboldx", "boldsymbol", "braket", "color", "textmacros"], "reject_unknown": false, "abort_on_error": false, "max_group_depth": 128, "lower_attributes": { "enabled": true }, "rewrite": { "enabled": false, "max_iterations": 100 }, "finalize_ast": { "enabled": true }, "flatten_groups": { "enabled": true, "preserve_rendered_spacing": true } } ``` `packages` lists the loaded set without duplicates in catalog order, regardless of the order requested. Key order within `resolved` is not significant. An unknown style, profile, or package, or an invalid override (including `rewrite.max_iterations = 0`, even when rewriting is disabled) fails with `-32602` before the configuration is registered. #### normalize [#normalize] Normalizes `latex` with the configuration stored under `config`. The result is identical to what a `TransformEngine` returns for the same input and effective config. A formula that fails to normalize is a normal outcome, reported as error code `1` (see [Errors](#errors)). An unknown `config` id fails with `-32602`. With `report: true`, the response also carries the transform report, including `warnings` for unknown input commands and environments. The report uses snake_case, as in `normalize --json --report`. The plain response omits `report`, and requesting it does not change output or error handling. With `timing: true`, the response also carries per-stage durations (see [Timing](#timing)). #### shutdown [#shutdown] Returns `null`. Every later request fails with `-32600`, and the server then only waits for end of file on stdin. ### Errors [#errors] Error responses carry a JSON-RPC error object `{"code","message","data"?}`. There are two layers: protocol errors mean the request itself was invalid, while code `1` is a per-formula result that callers should treat as data. | `code` | Condition | `data` | | -------- | ------------------------------------------------------------------------------------------------------ | ---------------------------------- | | `-32700` | The line is not valid UTF-8 JSON; the response `id` is `null` | none | | `-32600` | Not a valid JSON-RPC 2.0 request, a batch array, or any request after `shutdown` | none | | `-32601` | Unknown method | none | | `-32602` | Malformed params, unknown style, profile, or package, invalid `overrides`, or unconfigured `config` id | `{"message"}` | | `-32002` | Request before `initialize` | none | | `1` | The formula failed to normalize | `{"kind","diagnostics","timing"?}` | * For `-32602`, `message` is `"Invalid params"` and `data.message` explains the problem, including the path of an offending field such as `config.overrides.rewrite.enabld`. * For code `1`, `message` describes the failure and `data.kind` classifies it as `parse` (the formula did not parse into a complete tree), `transform` (a normalization step failed), or `internal` (an internal error, including a panic inside the server; the server stays usable after a panic). * `data.diagnostics` lists parse diagnostics for `parse` failures and is empty otherwise. Each diagnostic has `kind` (a stable kebab-case identifier or `null`), `message`, `span` (`start` and `end` byte offsets into `latex`), `expected`, `found`, and `contexts`. * The server applies no timeouts. A client that needs one should kill and restart the process. ### Timing [#timing] With `timing: true`, a successful `normalize` result contains `timing` with three members, each an integer number of nanoseconds: | Member | Stage | | -------------- | ------------------------------------ | | `parse_ns` | Parsing `latex` into a complete tree | | `transform_ns` | Normalizing the tree | | `serialize_ns` | Serializing the normalized tree | * Timings cover only the server's internal work, not JSON decoding, encoding, or I/O. * A code `1` failure carries `timing` in `error.data.timing` with only the stages that ran: a parse failure has only `parse_ns`, and a transform failure has `parse_ns` and `transform_ns`. Failures caused by a panic carry no timing. * Requesting timing never changes `output` or the error classification. * Whether a server supports timing is decided from successful responses; an implementation without timing support ignores the member. ### Versioning [#versioning] * `protocolVersion` increases only when method semantics or required fields change incompatibly. New optional fields and new methods keep the version. * Clients must refuse to continue with a `protocolVersion` they do not recognize. * Servers ignore unknown members of `params`, so clients can send newer optional fields to older servers. Unknown members of `config` and `overrides` are rejected instead, so a configuration an older server cannot honor fails at `configure` rather than silently producing different results. ## Next steps [#next-steps] - [Transforms](https://texform.dev/docs/transforms) — The four profiles and the multi-phase pipeline behind normalize. - [Normalize a corpus](https://texform.dev/docs/normalize-a-corpus) — Run one reused engine over a dataset from Python or JavaScript. - [Installation](https://texform.dev/docs/installation) — Install the Python, JavaScript, and Rust libraries. --- # Docs for AI tools Give coding assistants and agents the TeXForm docs as Markdown: a .md version of every page, plus llms.txt and llms-full.txt at the site root. https://texform.dev/docs/ai-tools Every page on this site is also published as plain Markdown, and two text files at the site root index those pages for language models. Point a coding assistant or agent at them instead of the HTML pages: they carry the same content without navigation or scripts, and formulas appear as LaTeX source rather than rendered images. ## Markdown pages [#markdown-pages] Append `.md` to a page URL to get its Markdown version. The home page is the one exception: its Markdown version is `/index.md`. | Page | Markdown version | | ----------------------------------------- | -------------------------------------------- | | `https://texform.dev/` | `https://texform.dev/index.md` | | `https://texform.dev/docs` | `https://texform.dev/docs.md` | | `https://texform.dev/docs/transforms` | `https://texform.dev/docs/transforms.md` | | `https://texform.dev/api/python` | `https://texform.dev/api/python.md` | | `https://texform.dev/latex/commands/frac` | `https://texform.dev/latex/commands/frac.md` | | `https://texform.dev/rules` | `https://texform.dev/rules.md` | On documentation and reference pages, **Copy page** next to the breadcrumbs copies the Markdown version to the clipboard, and **View Markdown** below the table of contents opens it. Each HTML page also names its Markdown version in a `` element, so tools that read the page head can find it. Each Markdown page starts with the page title, its one-line description, and the URL of the HTML page. Links to other pages on the site are absolute URLs. The rest depends on the page: * **Documentation**: the page text. Tabs, cards, and callouts are written out as plain Markdown, diagrams as a sentence or a Mermaid code block. The changelog and the renderer coverage tables are included as Markdown. * **API reference**: each class, function, and method with its signature, parameters, return value, raised errors, and examples. * **LaTeX math reference**: for a command or environment, its packages and allowed mode, the argument specification in each package, examples, renderer support in MathJax, KaTeX, and XeTeX, related rewrite rules, and renderer test cases. Symbol pages give the glyph and Unicode code point; package pages list their commands and environments with signatures and renderer support. * **Rewrite rules**: level, fidelity, enabling packages, profiles, rewrite patterns, before-and-after examples, the LaTeX entries the rule triggers on, eliminates, and produces, and code to disable it. * **Index pages**: `/latex/commands.md`, `/latex/environments.md`, `/latex/symbols.md`, `/latex/packages.md`, and `/rules.md` list every entry with a link to its page. ## llms.txt [#llmstxt] [`/llms.txt`](https://texform.dev/llms.txt) follows the [llms.txt](https://llmstxt.org/) convention. It opens with a short description of TeXForm, the install commands, the four normalization profiles, and where to start reading, then links to the Markdown version of the site's pages in these sections: * **Get started**, **Concepts**, **Guides**, and **Project**: the documentation pages in sidebar order, with the documentation overview first under Get started. * **API Reference**: the overview, then every Python and JavaScript page. * **LaTeX Math Reference**: the section overview, the command, environment, symbol, and package indexes, and renderer support. Individual entries are not listed; the Markdown index pages link to each of them. * **Rewrite Rules**: the rule index, which gives every rule with its level and fidelity. * **Optional**: the Internals pages and a text version of the home page. ## llms-full.txt [#llms-fulltxt] [`/llms-full.txt`](https://texform.dev/llms-full.txt) is a single file with the same opening as `llms.txt`, followed by the Markdown of every documentation page in sidebar order and then the whole API reference: the overview, the Python pages, and the JavaScript pages. Pages are separated by horizontal rules. It does not include the LaTeX math reference or the rewrite rules; fetch those pages individually. The file is large, so when context is limited, start from `llms.txt` and fetch only the pages a task needs. ## Point an assistant at the docs [#point-an-assistant-at-the-docs] For a single question in a chat, use **Copy page** on the relevant page and paste it with your question. For an agent that can fetch URLs, give it `llms.txt` as the starting point: ```text Read https://texform.dev/llms.txt and fetch the Markdown pages you need. Then show how to normalize a list of formulas with the corpus profile in Python. ``` The files are plain HTTP resources, so scripts can fetch them directly: ```sh curl -s https://texform.dev/latex/commands/frac.md ``` The Markdown and text files are generated in the same build as the HTML pages and describe the same TeXForm version, the latest release. If you use an older version, check the [changelog](https://texform.dev/docs/changelog) for differences. --- # What TeXForm is TeXForm parses, edits, and normalizes LaTeX math against a command knowledge base. What it does, what it leaves to renderers and CAS tools. https://texform.dev/docs/what-is-texform Programs that process LaTeX formulas — cleaning OCR training data, comparing formulas for equivalence, preparing pretraining corpora, building math editors — keep reinventing the same fragile layer: regular expressions and one-off heuristics that guess what `\frac`, `\over`, or `\quantity` mean. The hard part was never the string manipulation. It is knowing the argument structure of every command well enough to work on formulas as syntax trees instead of text. TeXForm is that layer. It parses a formula against a structured knowledge base of command and environment specifications from the supported LaTeX packages, hands you an editable tree, and normalizes formulas into canonical forms tuned for specific downstream uses. One Rust core, available in Rust, Python, and JavaScript. Thirty seconds of it looks like this — a `physics` macro that only renders with that package loaded, turned into a universal form that any renderer accepts. `physics` is not in the default package set, so the engine's knowledge base names it: **Python** ```python import texform kb = texform.KnowledgeBase(packages=["base", "ams", "physics"]) engine = texform.TransformEngine(profile="corpus", knowledge_base=kb) print(engine.normalize(r"\dv{f}{x}")) ``` ```text \frac { \mathrm { d } f } { \mathrm { d } x } ``` **JavaScript** ```js import { KnowledgeBase, TransformEngine } from 'texform'; const knowledgeBase = new KnowledgeBase({ packages: ['base', 'ams', 'physics'] }); const engine = new TransformEngine({ profile: 'corpus', knowledgeBase }); console.log(engine.normalize('\\dv{f}{x}')); ``` ```text \frac { \mathrm { d } f } { \mathrm { d } x } ``` ## Three pillars [#three-pillars] Everything in TeXForm sits on three assets, and each one is a page in these docs: * **A knowledge-driven parser.** Formulas parse into real syntax trees because the parser reads each command's argument signature from a curated knowledge base rather than guessing from syntax. Unknown commands and unparseable fragments survive as explicit nodes under your control, never as a crash. See [Parsing](https://texform.dev/docs/parsing). * **An editable document tree.** Parsing produces a `Document`: a DOM-style tree you query, mutate, and serialize back to LaTeX. Edits are validated and return structured errors, and the serializer guarantees text idempotency, so its output parses and serializes back to the same string. See [Serialization](https://texform.dev/docs/serialization). * **A profile-based transform engine.** Normalization has no single correct answer, so TeXForm never imposes one true form. You pick a profile that names your downstream use — authoring, render-faithful, corpus, or equivalence — and a curated rule pipeline produces the canonical form for that use. Every rule declares its rule level, its worst-case fidelity, and the forms it eliminates, and the engine enforces that contract after every run. See [Transforms](https://texform.dev/docs/transforms). Two of those assets did not exist as reusable artifacts before: the argspec knowledge base, whose signatures are validated by rendering against MathJax, KaTeX, and XeTeX rather than transcribed from documentation, and the curated rewrite rule set with its enforced eliminated-form contract. ## The unit of work is a formula [#the-unit-of-work-is-a-formula] TeXForm works on math formulas, not on LaTeX documents. There is no notion of sections, bibliographies, floats, `\usepackage`, or a preamble; you hand it the contents of a math environment and it hands back the contents of a math environment. Packages are how the knowledge base is organized, not something your input declares. That is a scope decision, not a limitation to be lifted later. Whole-document conversion is a genuinely different problem with different tools — LaTeXML and plasTeX live there. Keeping the unit of work at formula scale is what makes it reasonable to demand a real knowledge base for every command, an idempotency guarantee on the output, and contract checking on every rewrite. ## Why not an existing tool [#why-not-an-existing-tool] The tools people already reach for each cover part of this job: * **General LaTeX parsers** give you a tree, but decide command arguments from a built-in table, from signatures you write, or by attaching the groups that follow, so package commands such as `\dv{f}{x}` often come out as a name followed by loose groups. * **Renderers** know commands well enough to display them, but produce HTML, SVG, or MathML rather than LaTeX you can store, edit, or compare. * **Normalization scripts** in OCR and evaluation code rewrite tokens or strings, so they pass unknown commands through unchanged and cannot tell you when a formula was not understood. [How TeXForm compares](https://texform.dev/docs/comparison) goes through these tools job by job, including when one of them is the better choice. ## Where renderers and CAS tools fit [#where-renderers-and-cas-tools-fit] TeXForm's relationship to MathJax and KaTeX is easy to misread, so it is worth stating both directions. **Upstream, they validate the knowledge base.** Every argspec is checked by rendering it against MathJax, KaTeX, and XeTeX, which is how an entry earns the coverage status you see on the [LaTeX reference](https://texform.dev/latex) pages. That validation is internal tooling; it is not something TeXForm does at runtime. See [Renderer validation](https://texform.dev/docs/renderer-validation). **Downstream, they display the output.** TeXForm emits a LaTeX string, and a renderer turns that string into something a reader sees. It does not embed, wrap, or replace a renderer, and there is no plugin to install. The practical connection is that `Faithful` and `Corpus` normalization expand package-specific macros into universal forms, so, with the `physics` package loaded, a formula written with commands such as `\dv` or `\ket` becomes a formula a vanilla MathJax or KaTeX deployment can render with no extra packages. For mathematical meaning, the partner is a computer algebra system. The `Equiv` profile canonicalizes equivalent *spellings* to the same string, which resolves a large share of notation noise with a plain string comparison — but a formula whose equivalence is mathematical rather than notational needs SymPy or a similar tool. [Compare formulas](https://texform.dev/docs/compare-formulas) shows that two-layer workflow. ## What TeXForm does not do [#what-texform-does-not-do] Three boundaries are worth stating plainly, because each one saves you from asking TeXForm for the wrong thing: * **It does not compute mathematical equivalence.** TeXForm canonicalizes notation; it does not know that `x + x` equals `2x`. Normalize with TeXForm first, then hand the result to a CAS. * **It does not render.** TeXForm emits LaTeX, never pixels, HTML, or MathML. Fidelity levels are contracts its rules declare, not comparisons it runs. * **It is not a document processor.** The unit of work is a formula. Sections, bibliographies, and floats are out of scope by design. Use TeXForm when you need to parse, edit, and normalize LaTeX math as structured syntax with command-level knowledge, and compose it with a CAS or a renderer when your task crosses into mathematical meaning or visual output. - [Quick start](https://texform.dev/docs/quick-start) — Install it and normalize your first formula in a few minutes. - [Pipeline](https://texform.dev/docs/pipeline) — The path a formula takes, from source text to its four exits. - [Glossary](https://texform.dev/docs/glossary) — Profile, rule level, SyntaxNode, overlay — the terms used throughout. - [How TeXForm compares](https://texform.dev/docs/comparison) — Parsers, renderers, CAS tools, and normalization scripts, compared job by job. --- # How TeXForm compares TeXForm next to LaTeX parsers, normalization scripts, renderers, and CAS tools: a feature matrix for the tools that overlap with it, and when to use which. https://texform.dev/docs/comparison Any program that reads LaTeX math as more than a string needs to know what each command takes. `\frac12` has two arguments, `\binom{n}{k}` has two, and `\dv{f}{x}` has two only when the `physics` package is in play. Several kinds of tool carry some of that knowledge, and TeXForm overlaps with each of them in a different way. ## The landscape [#the-landscape] | Tool | Kind | What you get | Overlap with TeXForm | | --------------------------------------- | -------------------- | ---------------------------------------------- | ----------------------------------------- | | pylatexenc, unified-latex, TexSoup | LaTeX parser | A tree of a LaTeX document, and LaTeX back out | Parsing, editing, serialization | | Compute Engine | Math engine | MathJSON expressions, and LaTeX back out | Parsing and serialization, as mathematics | | SymPy, Math-Verify | Math engine | SymPy expressions for checking answers | Comparing formulas, as mathematics | | im2markup normalizer | Normalization script | A token string for recognition labels | Normalization | | MATH `strip_string`, Minerva normalizer | Normalization script | A comparison key for grading answers | Normalization | | KaTeX, MathJax, Temml | Renderer | HTML and MathML | Knowing what commands mean | | texvcjs | Validator | LaTeX checked against a fixed command list | Knowing what commands mean | The parsers and Compute Engine are libraries that read LaTeX into a structure and write LaTeX back, which is what TeXForm does too, so the [feature comparison](#feature-comparison) below puts them side by side. Normalization scripts, renderers, and math engines solve narrower jobs; the sections after the matrix cover how TeXForm fits next to them. ## Feature comparison [#feature-comparison] | | TeXForm | pylatexenc | unified-latex | TexSoup | Compute Engine | | ----------------------------------------------- | ----------------------------------------------------------------------- | -------------------------------------------------------------------- | --------------------------------------------------------------------- | -------------------------------------------------------------- | ---------------------------------------------------------------------- | | **Parsing** | | | | | | | Math command signatures | Yes: Knowledge base, by package | Partly: LaTeX core | Partly: Core and selected packages | No: Takes following groups | Partly: Math dictionary | | `amsmath` commands, such as `\binom` | Yes | No: Groups left as siblings | No: Groups left as siblings | Partly: By position | Partly: As mathematics | | Package macros, such as `\dv` | Yes: Opt-in `physics` | No: Groups left as siblings | No: Groups left as siblings | Partly: By position | No: Error node | | Unbraced arguments, `\frac12` | Yes | Yes | Yes | No: Text `12` | Yes | | Infix commands, `a \over b` | Yes: Infix node | No: Plain command | No: Plain command | No: Plain command | Yes: As division | | Your own command signatures | Yes | Yes: Macro specs | Yes: Macro specs | No: No public API | Yes: Dictionary entries | | **Errors** | | | | | | | Reports unknown commands | Yes: Rejected or marked | No | No | No | Yes: Error node | | Reports malformed input, `\frac{a}{b` | Yes: Diagnostics | Partly: With tolerant parsing off | No | Yes: Exception | Yes: Error node | | **Output and editing** | | | | | | | Writes LaTeX back | Yes: Canonical spacing | Yes: Source text | Yes: Light reformatting | Yes: Source text | Partly: Notation may change | | Editable tree | Yes: Edits checked against signatures | Partly: Write-back in 3.0 beta | Yes: unist tree | Yes: Insert, replace, delete | Partly: Immutable expressions | | **Normalization** | | | | | | | Folds equivalent spellings | Yes: Four profiles | No | No | No | Partly: Mathematical canonical form | | Expands package macros to core LaTeX | Yes | No | Partly: Definitions you supply | No | No | | **Scope** | | | | | | | Whole LaTeX documents | No: Formulas only | Yes | Yes | Yes | No: Formulas only | | Languages | Rust, Python, JavaScript, CLI | Python | JavaScript | Python | JavaScript | Some cells need a qualifier: * **TeXForm** loads a default package set; `physics` is opt-in because it gives `\ket` and related commands different meanings than `braket` does. Strict parsing rejects unknown commands, and the default lenient mode keeps them as nodes marked unknown. Package macros are expanded by the `Faithful` profile and stronger ones; `Authoring` keeps them. See [Transforms](https://texform.dev/docs/transforms) for the profiles. * **pylatexenc** 2.x writes back each node's source text, so changing a node does not change the output; the 3.0 beta adds a recomposer that rebuilds LaTeX from the nodes. * **unified-latex** ships signatures for the LaTeX core and selected packages, including `mathtools` but not `amsmath` or `physics`. Its `unified-latex-util-macros` package expands macros once you give it their definitions. * **TexSoup** attaches every bracket and brace group that follows a command, apart from a small hard-coded table, so `\binom{n}{k}` and `\dv{f}{x}` get two arguments while `\frac12` gets none. * **Compute Engine** reads formulas as mathematics: `a \over b` comes back as `\frac{a}{b}`, and an `aligned` environment as a list. Its canonical forms may change between versions. The number of arguments each parser attached on the inputs above, with groups left as siblings in parentheses: | Input | pylatexenc | unified-latex | TexSoup | TeXForm | | -------------- | ------------ | ------------- | ------------- | ------- | | `\dv{f}{x}` | 0 (2 groups) | 0 (2 groups) | 2 | 2 | | `\frac12` | 2 | 2 | 0 (text `12`) | 2 | | `\binom{n}{k}` | 0 (2 groups) | 0 (2 groups) | 2 | 2 | **When a parser is the right choice.** For whole LaTeX documents, with sections, environments, and preambles, use pylatexenc, unified-latex, or TexSoup; TeXForm parses formulas only. unified-latex also fits naturally if your pipeline already lives in the unified.js ecosystem. For formulas, TeXForm gives you the argument structure without writing signatures, and an editable [`Document`](https://texform.dev/docs/edit-a-document) whose edits are validated against the same signatures. ## Normalization scripts [#normalization-scripts] Many recognition datasets and metrics normalize labels with a copy of the im2markup preprocessing script, which parses with a vendored KaTeX 0.6.0 and re-serializes the tree as space-separated tokens. Answer-grading code strips and rewrites strings until two answers can be compared: `strip_string` from the MATH benchmark, the normalization in lm-evaluation-harness adapted from the Minerva paper, and evaluation scripts derived from them. TeXForm does the same jobs from the parse tree instead of string edits. The `Corpus` profile produces complete canonical forms intended as training labels, and `Equiv` folds equivalent spellings into one string for comparison and deduplication. Every rule declares the forms it eliminates, and the engine checks that they are gone. Malformed formulas fail with diagnostics instead of producing a different string, so you can sort a batch into normalized, repairable, and discarded formulas; [Clean OCR/MER output](https://texform.dev/docs/clean-ocr-mer-output) shows the workflow. Serialization in either style is text-idempotent: parsing a normalized formula and serializing it again returns the same string. | Input | im2markup normalizer | TeXForm `Corpus` | | ---------------------- | --------------------------- | ------------------------------- | | `a \over b` | `\frac { a } { b }` | `\frac { a } { b }` | | `{\bf x} + \mathbf{y}` | `{ \bf x } + \mathbf { y }` | `\mathbf { x } + \mathbf { y }` | | `a \le b \leq c` | `a \le b \leq c` | `a \leq b \leq c` | For `\dv{f}{x}`, the im2markup normalizer keeps `\dv { f } { x }`, while `Corpus` expands it into a fraction of differentials. TeXForm's spaced token output matches what tokenized training labels usually expect; if you want compact LaTeX instead, see [Configure output style](https://texform.dev/docs/configure-output-style). **When the script is the right choice.** If you must reproduce numbers reported on an existing benchmark, use that benchmark's own normalizer, because changing it changes the scores. TeXForm is the choice for new labels and new pipelines. ## Renderers and math engines [#renderers-and-math-engines] **Renderers** such as KaTeX, MathJax, and Temml know commands well enough to display them, and emit HTML or MathML, not LaTeX. Support for package macros varies: MathJax renders `physics` macros once its `physics` extension is loaded, which it is not by default; Temml bundles physics support; with KaTeX you define the macros yourself in its `macros` option. TeXForm's `Faithful` profile expands convenience macros into core LaTeX while keeping the rendered formula the same, so `\dv{f}{x}` becomes `\frac { \mathrm { d } f } { \mathrm { d } x }` and works on a stock renderer; see [Use TeXForm with MathJax or KaTeX](https://texform.dev/docs/use-with-a-renderer). If you control the renderer and only need display, loading the extension is the simpler path. **texvcjs**, the checker behind math on Wikipedia, accepts LaTeX from a fixed table of functions and rejects anything else. It suits a site that needs one safe input language; TeXForm's knowledge base is curated per package and you choose which packages to load. **Math engines** such as SymPy and Math-Verify read formulas as mathematics: `\frac12` becomes the number one half. That is the right tool for checking an answer, and the wrong one for keeping notation. TeXForm does not decide that `x + x` equals `2x`; for mathematical equivalence, normalize with TeXForm first and hand the result to SymPy or Math-Verify. [Compare formulas](https://texform.dev/docs/compare-formulas) shows the two layers together. ## When to use something else [#when-to-use-something-else] * **Displaying math:** KaTeX, MathJax, or Temml. TeXForm emits LaTeX and never renders. * **Symbolic math and answer checking:** SymPy, Math-Verify, or Compute Engine. * **Whole documents:** LaTeXML, pandoc, or plasTeX convert full LaTeX documents to HTML, XML, or other formats. * **Formatting `.tex` source:** tex-fmt or latexindent. * **Converting to Typst or MathML:** MiTeX or tex2typst for Typst, Temml or texmath for MathML. ## What TeXForm does not cover [#what-texform-does-not-cover] * **Commands outside the knowledge base.** The knowledge base is curated per package; browse what it covers in the [LaTeX reference](https://texform.dev/latex). Strict parsing rejects anything else, and lenient parsing keeps it as an opaque node. You can register your own commands; see [Inject custom commands](https://texform.dev/docs/inject-custom-commands) and [Handle parse errors](https://texform.dev/docs/handle-parse-errors). * **Rewrites nobody has written yet.** Normalization covers the rules in the [rule reference](https://texform.dev/rules); a command the knowledge base parses is not necessarily rewritten by any profile. * **Mathematical equivalence, rendering, and documents.** These are out of scope by design; [What TeXForm is](https://texform.dev/docs/what-is-texform) explains why. ## Tools compared [#tools-compared] Project pages: [pylatexenc](https://github.com/phfaist/pylatexenc), [unified-latex](https://github.com/siefkenj/unified-latex), [TexSoup](https://github.com/alvinwan/TexSoup), [im2markup](https://github.com/harvardnlp/im2markup), [MATH](https://github.com/hendrycks/math), [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness), [SymPy](https://docs.sympy.org/latest/modules/parsing.html), [Math-Verify](https://github.com/huggingface/Math-Verify), [Compute Engine](https://mathlive.io/compute-engine/), [texvcjs](https://github.com/wikimedia/texvcjs), [KaTeX](https://katex.org/), [MathJax](https://docs.mathjax.org/), [Temml](https://temml.org/), [LaTeXML](https://math.nist.gov/~BMiller/LaTeXML/), [pandoc](https://pandoc.org/), [plasTeX](https://github.com/plastex/plastex), [tex-fmt](https://github.com/WGUNDERWOOD/tex-fmt), [latexindent](https://github.com/cmhughes/latexindent.pl), [MiTeX](https://github.com/mitex-rs/mitex), [tex2typst](https://github.com/qwinsi/tex2typst), [texmath](https://github.com/jgm/texmath). If a tool has changed since, corrections are welcome: [open an issue](https://github.com/texform-dev/texform/issues/new). - [What TeXForm is](https://texform.dev/docs/what-is-texform) — The three pillars, and what TeXForm leaves to renderers and CAS tools. - [Transforms](https://texform.dev/docs/transforms) — The four profiles and the rule ladder behind them. - [Clean OCR/MER output](https://texform.dev/docs/clean-ocr-mer-output) — Sort recognition output into normalized, repairable, and discarded formulas. --- # Pipeline How a LaTeX formula travels through TeXForm, from source text through the parser and the document tree to its four exits. https://texform.dev/docs/pipeline A LaTeX formula enters TeXForm as a string and leaves through one of four exits. This page is the map: it shows the single path every formula takes and links each stage to the concept page that explains it in depth. ## What the pipeline is [#what-the-pipeline-is] TeXForm processes a formula in one direction. The parser turns source text into a `SyntaxNode`, that snapshot becomes an editable `Document`, and the `Document` exits as serialized text, tokenized text, a normalized formula, or a structured snapshot. LaTeX source → Parser and knowledge base → SyntaxNode snapshot → editable Document. The Document supports text output, typed tokens, normalization, and structured snapshots. The flow is single-direction: source becomes a snapshot, the snapshot becomes a working tree, and the working tree produces output. Nothing loops back on its own; you re-enter the pipeline by parsing a new string. ## The parser produces a snapshot first [#the-parser-produces-a-snapshot-first] The parser reads source text and consults a knowledge base of command and environment signatures to decide how each token binds its arguments. Its immediate output is a `SyntaxNode`: an immutable, lossless snapshot of the parse, which may contain `Error` nodes for fragments it could not parse. The snapshot is never edited in place. See [Parsing](https://texform.dev/docs/parsing) for why the parser needs a knowledge base and what the three possible parse results mean. ## SyntaxNode is the wire format, Document is the working format [#syntaxnode-is-the-wire-format-document-is-the-working-format] The most useful distinction in TeXForm is between its two public trees: * A `SyntaxNode` is the **wire format** — the snapshot you serialize to JSON, store, transport, or use as a test fixture. It carries no editing behavior. * A `Document` is the **working format** — a DOM-style tree you query, mutate, serialize, and transform. Reads go through lightweight node handles; edits are validated and return a structured error instead of panicking. TeXForm converts the snapshot into a `Document` for you on the parse path, and bridges back the other way on demand. The two directions are symmetric over every node kind, including `Error` and `Prime`. ```python import texform doc = texform.Parser().parse(r"x^2")["document"] snapshot = doc.to_syntax() # working format -> wire format restored = texform.Document.from_syntax(snapshot) # wire format -> working format print(restored.to_latex()) ``` ```text x ^ { 2 } ``` A rebuilt document is bound to a knowledge base like any other: `from_syntax` uses the shared default unless you pass one. `TransformEngine.transform` accepts it when that is the engine's own knowledge-base instance and rejects it as foreign otherwise, so pass `engine.knowledge_base()` when you rebuild a document you intend to normalize in place. ## Four exits [#four-exits] A `Document` exits in four ways, each with a different purpose: * **`to_latex()`** serializes the tree back to LaTeX text, spaced by default or compact on request. Even a tree containing `Error` nodes round-trips: each error re-emits its captured source snippet verbatim. See [Serialization](https://texform.dev/docs/serialization). * **`to_tokenized_latex()`** runs that same serializer traversal with a recorder attached, returning the identical LaTeX string plus typed tokens with byte spans into it. It is one traversal, not a second lexing pass. See [Tokenized output](https://texform.dev/docs/serialization#tokenized-output). * **`TransformEngine`** normalizes the formula into a canonical form chosen by a profile. Normalization is gated on a complete tree — a tree with errors cannot be normalized. See [Transforms](https://texform.dev/docs/transforms). * **`to_syntax()`** converts the tree back to a `SyntaxNode` for structured-data output. This is the only exit that produces serde-friendly structured data; there is no method named `serialize` on `Document`. - [What TeXForm is](https://texform.dev/docs/what-is-texform) — The product boundary: what TeXForm does and what it leaves to other tools. - [Parsing](https://texform.dev/docs/parsing) — Why parsing is knowledge-driven and what a parse result can contain. - [Transforms](https://texform.dev/docs/transforms) — Profiles and the normalization pipeline behind the transform engine. - [Serialization](https://texform.dev/docs/serialization) — The output channels, text idempotency, and output style. --- # Parsing Why TeXForm parses LaTeX against a knowledge base, which packages it loads by default, and what a parse result can contain. https://texform.dev/docs/parsing Parsing LaTeX is hard not because of string manipulation but because the meaning of a command depends on knowing its argument structure. TeXForm parses against a knowledge base of command signatures, so a formula becomes a real syntax tree instead of a best-effort guess. ## What parsing is [#what-parsing-is] Parsing turns LaTeX source into a `Document` — an editable tree — together with a list of diagnostics describing anything the parser could not handle. **Python** ```python import texform result = texform.Parser().parse(r"\frac{x}{y}") print(result["document"].to_latex()) ``` ```text \frac { x } { y } ``` **JavaScript** ```js import { Parser } from 'texform'; const result = new Parser().parse('\\frac{x}{y}'); console.log(result.document.toLatex()); ``` ```text \frac { x } { y } ``` ## Why parsing is knowledge-driven [#why-parsing-is-knowledge-driven] Argument structure is not something a grammar can infer from syntax alone. `\frac{x}{y}` takes two mandatory arguments, but nothing in the characters `\frac` says so — a generic LaTeX grammar cannot know that `\frac` consumes the next two groups while `\alpha` consumes none. The only way to parse `\frac`, `\over`, or `\quantity` correctly is to know each command's signature in advance. This is why regular expressions and one-off heuristics keep breaking on real formulas: they guess at structure the source does not encode. TeXForm reads each command's signature from a knowledge base, so the parser binds arguments the way the command actually behaves rather than guessing. ## Packages and context [#packages-and-context] The knowledge base is organized into packages. Each package contributes a set of known command and environment signatures, and you choose which packages the parser consults. Listing them shows the whole catalog: **Python** ```python import texform for pkg in texform.list_packages(): print(pkg["name"], pkg["commands"], pkg["environments"]) ``` ```text ams 35 28 base 270 7 bboldx 12 0 boldsymbol 1 0 braket 11 0 color 5 0 physics 182 1 textmacros 78 0 ``` **JavaScript** ```js import { listPackages } from 'texform'; for (const pkg of listPackages()) { console.log(pkg.name, pkg.commands, pkg.environments); } ``` ```text ams 35 28 base 270 7 bboldx 12 0 boldsymbol 1 0 braket 11 0 color 5 0 physics 182 1 textmacros 78 0 ``` ### The default package set [#the-default-package-set] `list_packages()` lists the catalog, not the parser's default selection. A `Parser` or `TransformEngine` built without a knowledge base shares the default one, which loads the default set: `base`, `ams`, `braket`, `textmacros`, `bboldx`, `boldsymbol`, and `color`. `physics` is opt-in because it redefines commands that `braket` also defines, and gives standard commands such as `\div`, `\Re`, and `\Im` its own meanings. Loading both is allowed: TeXForm imports built-in packages in a fixed order, with `physics` overriding the shared command definitions from `braket`. Reordering the `packages` array does not change that precedence. Select the package whose command syntax your input uses. The consequence is concrete. Under the default set, `\ket` resolves to the `braket` record, whose signature is `m`, and normalization keeps `\ket{\psi}` as written. The `physics` record for the same name has the signature `s m` — a starred variant plus one mandatory argument — and `Corpus` normalization expands `\ket{\psi}` into `\left \vert \psi \right \rangle`. If your corpus means `physics`, ask for it explicitly: **Python** ```python import texform kb = texform.KnowledgeBase(packages=["base", "ams", "physics"]) parser = texform.Parser(knowledge_base=kb) print(kb.lookup_command("ket", "math")["spec_string"]) print(kb.lookup_command("ket", "math")["from_packages"]) ``` ```text s m ['physics'] ``` **JavaScript** ```js import { KnowledgeBase, Parser } from 'texform'; const knowledgeBase = new KnowledgeBase({ packages: ['base', 'ams', 'physics'] }); const parser = new Parser({ knowledgeBase }); console.log(knowledgeBase.lookupCommand('ket', 'math').specString); console.log(JSON.stringify(knowledgeBase.lookupCommand('ket', 'math').fromPackages)); ``` ```text s m ["physics"] ``` Naming `packages` yourself replaces the default set rather than adding to it, so list every package you need, including `base`. When a command is not in your selected packages, you can register it yourself by passing context items to the knowledge base the parser uses. A context item is a small record carrying a command name and an argspec; the parser treats it as if it were a built-in signature. See [Inject custom commands](https://texform.dev/docs/inject-custom-commands) for the full workflow. ## Argument specifications [#argument-specifications] A command's signature is written as an **argspec** — a compact, xparse-style string describing each argument the command consumes. The argspec is what tells the parser to read two groups after `\frac` and none after `\alpha`. It is the single mechanism behind knowledge-driven parsing. The notation is small. A few common slots: * `m` — one mandatory argument. `\frac` is `m m` (two mandatory groups). * `o` — one optional bracketed argument. `\sqrt` is `o m` (an optional index, then the radicand). * `s` — an optional leading star. With `physics` loaded, `\ket` is `s m`, so `\ket*{\psi}` parses with its star recorded. You can validate an argspec string directly, which reports how many arguments it describes: **Python** ```python import texform print(texform.validate_argspec("o m")) ``` ```text {'valid': True, 'error': None, 'arg_count': 2, 'parsed': [{'required': False, 'no_leading_space': False, 'nullable': False, 'kind': {'type': 'content', 'mode': 'math'}, 'form': {'type': 'standard'}}, {'required': True, 'no_leading_space': False, 'nullable': False, 'kind': {'type': 'content', 'mode': 'math'}, 'form': {'type': 'standard'}}]} ``` **JavaScript** ```js import { validateArgspec } from 'texform'; const result = validateArgspec('o m'); console.log(result.valid, result.argCount); ``` ```text true 2 ``` The full notation — every slot kind, delimited forms, and TeXForm's extensions to xparse — is documented in the [argspec reference](https://texform.dev/docs/argspec). For why the notation is shaped the way it is and its known limits, see [Argspec design](https://texform.dev/docs/argspec-design). ## Configuring strictness [#configuring-strictness] Two independent knobs on `ParseConfig` decide how the parser reacts to trouble, and for both of them `True` means stricter. `reject_unknown` controls what happens to a command the knowledge base does not recognize; `abort_on_error` controls what happens when the parser hits a syntax error it cannot continue past. Both default to `False`, so the default configuration is lenient on both axes. They combine into four behaviors: | | `abort_on_error = False` | `abort_on_error = True` | | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | | `reject_unknown = False` | The default. Unknown commands survive as nodes; the parser recovers from syntax errors and collects every diagnostic. | Unknown commands survive; the first syntax error stops the parse and returns no tree. | | `reject_unknown = True` | Unknown names become diagnostics; recovery still applies to syntax errors. Recovery consumes the unknown command itself, so `\mycmd{x} + y` keeps `{x}`, `+`, and `y` alongside one `Error` node. | Both axes strict. Any unknown name or syntax error is a hard failure and the parse returns no tree. | The default corner is the most permissive, which is what you want when cleaning noisy input. The opposite corner — both knobs `True` — is what "strict parsing" means precisely: it is for input you expect to be clean and want to reject loudly. Rust names these two corners `ParseConfig::STRICT` and `ParseConfig::LENIENT`; Python and JavaScript express them through constructor defaults and per-call overlays instead, so there is no `ParseConfig.LENIENT` constant to import. Lenient parsing recovers where it can, but recovery is not a promise of a tree. A formula whose structure the parser cannot close, such as `\frac{a}{b`, still comes back with `document = None` under the default configuration, while `x^` comes back as a read-only tree holding an `Error` node. Treat "returned a tree" as something to check, not to assume — the three-state handling below is written for exactly that. The two knobs are orthogonal to the result's shape: `abort_on_error` is a parse-strictness switch, while whether the resulting tree contains errors is a separate, queryable property of the tree. For task-oriented recipes built on these settings, see [Handle parse errors](https://texform.dev/docs/handle-parse-errors) and [Clean OCR/MER output](https://texform.dev/docs/clean-ocr-mer-output). ## What parsing gives you [#what-parsing-gives-you] A parse result has exactly three honest states. TeXForm never fabricates a placeholder tree to pretend a document always exists: For x^2, the base x spans bytes [0, 1), the exponent 2 spans [2, 3), and the script spans [0, 3). Parsing yields no document, a clean editable document, or a partial read-only document with Error nodes. * **`None`** means a hard failure with no tree to return. It is reserved for that case and is never a synthesized error placeholder. * **A clean tree** (`has_errors()` is false) is a complete, editable `Document`. Empty input is not an error and not `None` — `""` parses to a `Document` holding a single empty root, the same legal state as `Document()`. * **A partial tree** (`has_errors()` is true) preserved the unparseable parts as `Error` placeholder nodes. The tree is still structurally valid, but it is read-only. Diagnostics and `Error` nodes are related but not identical. On the parser path, every `Error` node is emitted alongside at least one diagnostic — so if the tree `has_errors()`, the diagnostics list is non-empty. The converse does not hold: diagnostics and `Error` placeholders are separate channels, so a diagnostic does not by itself make an otherwise editable tree read-only. The `Document` you get back is the working tree for everything downstream. Three behaviors define how you work with it: * **Reads use node handles.** Navigation (`parent`, `children`, `next_sibling`, `ancestors`) and content accessors return lightweight read-only handles. * **Edits return a result.** Mutations are validated eagerly and report a structured error at the point of misuse rather than panicking. * **Trees with errors are read-only.** If `has_errors()`, every editing method fails with a read-only error. The only use for an error tree is inspection. For the editing workflow, see [Edit a document](https://texform.dev/docs/edit-a-document); for the exact edit-method semantics and error variants, see the [Document API](https://texform.dev/api/python/document). The deeper tree model — the internal arena beneath `Document` and the `from_syntax`/`to_syntax` symmetry — is covered in [Architecture](https://texform.dev/docs/architecture). - [Transforms](https://texform.dev/docs/transforms) — Normalize a parsed formula with profiles and the rule engine. - [Serialization](https://texform.dev/docs/serialization) — Turn a Document back into LaTeX text or a structured snapshot. - [Handle parse errors](https://texform.dev/docs/handle-parse-errors) — A complete pattern for the three parse-result states. - [Glossary](https://texform.dev/docs/glossary) — SyntaxNode, Document, context item, lenient and strict — defined in one place. --- # Reading argspecs The notation TeXForm uses to describe LaTeX command and environment argument signatures, with one real command per parameter kind. https://texform.dev/docs/argspec An argspec is the short string that describes the argument structure of a LaTeX command or environment. It records, in order, what each argument slot looks like and how the parser reads it: a mandatory braced group, an optional bracketed value, a leading star, a delimiter-wrapped argument, and so on. For `\frac{a}{b}` the argspec is `m m` — two mandatory arguments; for `\sqrt[n]{x}` it is `o m` — one optional argument followed by one mandatory argument. You encounter argspecs in two places. Every command and environment detail page shows the argspec for that name, so you can see exactly what it accepts. And when you teach TeXForm a command it does not know — by injecting a context item into the parser — you write the argspec yourself. This page is the reference for what each notation means. To check a string before using it, pass it to `validate_argspec` (see the links at the end). The notation follows the [xparse](https://ctan.org/pkg/xparse) argument-specifier convention, with a few additions described under [TeXForm extensions](#texform-extensions). The design rationale lives in [Argspec design](https://texform.dev/docs/argspec-design); this page covers only what the notation means and how it parses. ## Notation at a glance [#notation-at-a-glance] An argspec is a whitespace-separated sequence of slots. Each slot is one base token, optionally preceded by `!` and followed by a value-kind annotation: ```text [!] token [:kind] [?] ``` | Notation | Slot | Reads | | ----------------------------- | ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `m` | mandatory | A braced group `{...}` or a single token. | | `m{}` | mandatory braced group | A `{...}` group; errors if the next token is not `{`. | | `o` | optional | A bracketed `[...]` value if present. | | `s` | optional star | A leading `*` if present. | | `g` | optional braced group | A `{...}` group if the next token is `{`. | | `r⟨open⟩⟨close⟩` | required delimited | Content wrapped by the given delimiters; errors if `open` is absent. | | `d⟨open⟩⟨close⟩` | optional delimited | Content wrapped by the given delimiters if `open` is present. | | `r…` | required delimiter choice | Content wrapped by the first matching delimiter pair; errors if none match. | | `d…` | optional delimiter choice | Same, but skipped if no pair matches. | | `O{…}` `G{…}` `R…{…}` `D…{…}` | default-bearing aliases | The same as `o` / `g` / `r` / `d`; the `{…}` default is recorded for documentation and ignored when parsing. | | `:T :O :D :N :L :I :K :C` | value-kind annotation | Overrides how the slot's content is interpreted (text, operator-name content, delimiter, control-sequence name, dimension, integer, key–value, column template). | | `?` | nullable | Allows the slot to carry an empty value, e.g. `m:D?`. | | `!` | no leading space | The parser does not skip whitespace before testing for this slot. | The base token's letter case carries meaning: lowercase `o`/`g`/`r`/`d` are the plain forms, and the uppercase `O`/`G`/`R`/`D` aliases attach a default value that TeXForm records but does not act on. A mandatory slot defaults to math-mode content with no annotation. ## Mandatory arguments: `m` [#mandatory-arguments-m] A standard required argument. The parser reads a braced group `{...}`, or a single following token if there is no brace. `\frac` has the argspec `m m`: ```latex \frac{a}{b} ``` ```text \frac { a } { b } ``` Both arguments are mandatory, so a missing brace is a parse error rather than a skipped slot. With no value-kind annotation, the content is parsed in math mode. ## Mandatory braced groups: `m{}` [#mandatory-braced-groups-m] A required argument that must be written as `{...}`. Unlike plain `m`, it never falls back to reading a single bare token, and the braces are treated as structural grouping rather than visible delimiters. `\abs` in the `physics` package has the argspec `s m{}`: ```latex \abs{x} ``` ```text \abs { x } ``` The `s` slot absorbs an optional `*`; the `m{}` slot requires the body to be braced. ## Optional arguments: `o` [#optional-arguments-o] A standard optional argument. If the next token is `[`, the parser reads up to the matching `]`; otherwise the slot is left empty and parsing continues. `\sqrt` has the argspec `o m`: ```latex \sqrt[3]{x} ``` ```text \sqrt [ 3 ] { x } ``` Without the bracket, the optional slot is simply absent: ```latex \sqrt{x} ``` ```text \sqrt { x } ``` ## Optional star: `s` [#optional-star-s] Tests whether the next token is `*` and consumes it if so. A command only reacts to a trailing `*` if its argspec declares an `s` slot; otherwise the `*` is treated as ordinary content. `\ket` in the `physics` package has the argspec `s m`: ```latex \ket*{\psi} ``` ```text \ket* { \psi } ``` Without the star, the slot records its absence and the rest of the argument list is read unchanged. ## Optional braced group: `g` [#optional-braced-group-g] Reads a `{...}` group only when the next token is `{`. This is how commands take a variable number of trailing braced arguments: each extra `{...}` fills one more `g` slot. `\braket` in the `physics` package has the argspec `s m g`: ```latex \braket{a}{b} ``` ```text \braket { a } { b } ``` The first `{a}` fills the mandatory `m`; the second `{b}` fills the optional `g`. With only one braced group, the `g` slot stays empty. ## Delimited arguments: `r` and `d` [#delimited-arguments-r-and-d] A delimited slot reads content wrapped by an explicit open/close delimiter pair instead of braces or brackets. `r` makes the wrapping required (a missing open delimiter is an error); `d` makes it optional (a non-matching next token skips the slot). The delimiters can be single characters or control sequences such as `\langle`. A single-pair delimited form is written `d()` or `r()`. For an injected command with argspec `r()`, the argument must be parenthesized: ```latex \wrapped(x) ``` ```text \wrapped ( x ) ``` The matched delimiters are recorded on the argument, so the original wrapping round-trips through serialization rather than being normalized to braces. ## Delimiter choice: `r<…>` and `d<…>` [#delimiter-choice-r-and-d] When a single argument slot accepts several different delimiter shapes, the delimiters are listed as `` pairs. The parser tries each pair in order and reads with the first one that matches. This is one slot, not several: every shape feeds the same argument position. `\grad` in the `physics` package has the argspec `d<(,)><[,]><{,}>`: ```latex \grad(\phi) ``` ```text \grad ( \phi ) ``` Because `\grad`'s delimiter slot is optional (`d`), it can also appear with no following delimiter at all, in which case the slot is skipped: ```latex \grad\phi ``` ```text \grad \phi ``` The characters `<`, `>`, and `,` are reserved inside the pair list and cannot themselves be delimiters. ## Value-kind annotations: `:T`, `:O`, `:D`, `:N`, `:L`, `:I`, `:K`, `:C` [#value-kind-annotations-t-o-d-n-l-i-k-c] By default a content slot is parsed as math-mode content. A `:` annotation after the token overrides how that slot's value is interpreted: | Annotation | Slot value | | ---------- | ---------------------------------------------------------------------------------------------------- | | *(none)* | Math-mode content. | | `:T` | Text-mode content. | | `:O` | Operator-name content: parsed as math content, but marked so operator-name runs serialize compactly. | | `:D` | A delimiter token. | | `:N` | A control-sequence name (a bare name string; escape sequences are rejected). | | `:L` | A dimension / length value. | | `:I` | An integer value. | | `:K` | A key–value list. | | `:C` | A column template (such as `c\|c` or `p{1em}`). | `\label` has the argspec `m:N` — a mandatory argument read as a name string: ```latex \label{sec:intro} ``` ```text \label { sec:intro } ``` The annotation attaches to the base token, so it combines with any form: `o:L` is an optional dimension, `m{}:T` is a required text-mode braced group, and `r():K` is a required parenthesized key–value argument. `\operatorname` has the argspec `s m:O`: an optional star followed by operator-name content. The `:O` annotation is distinct from the uppercase `O{default}` alias; `m:O` is still mandatory, while `O{foo}:O` would be an optional slot with a documented default. ## TeXForm extensions [#texform-extensions] The base letters `m`, `o`, `s`, `g`, `r`, and `d` carry their xparse meanings. The following notations are TeXForm additions, introduced because describing LaTeX **math** commands precisely needs information that xparse does not record: * **Value-kind annotations (`:T`, `:O`, `:D`, `:N`, `:L`, `:I`, `:K`, `:C`).** xparse does not distinguish what kind of value an argument holds; TeXForm needs that distinction to parse operator-name content, delimiters, dimensions, names, and column templates correctly rather than as generic content. * **The nullable modifier `?`.** Appended to an annotation, it allows the slot to carry an empty value while still being present — for example `m:D?` is a mandatory delimiter slot that may be written as empty `{}`, normalized to "no delimiter". It is accepted on every value kind except star and column. * **The required braced group `m{}`.** xparse's `g` is optional only; `m{}` is the required counterpart, where the braces are structural grouping rather than visible delimiters. * **Delimiter-choice pair lists (`r<…>`, `d<…>`).** A single `r` or `d` in xparse names exactly one delimiter pair. The `` list lets one slot accept several delimiter shapes — the form used by commands like `\grad` and `\qty` that wrap their argument in `()`, `[]`, `{}`, or `||` interchangeably. The `s` token is also treated as an ordinary optional slot that may carry the `!` no-leading-space prefix (as in the `\\` argspec `!s !o:L`), rather than as a special standalone marker. A few xparse tokens are intentionally not part of this notation, because no supported math command needs them: `l`, `u`, `t`, `e`/`E` embellishments, `v` verbatim, the `b` body specifier (an environment's body mode is tracked separately, not in its argspec), and the `+` long-argument prefix. ## Validate an argspec [#validate-an-argspec] To confirm a string parses and to inspect the resulting slots, use the validate-argspec function. It returns whether the string is valid, an error message if not, the slot count, and the parsed structure. **Python** ```python import json import texform print(json.dumps(texform.validate_argspec("o m"), indent=2)) ``` ```text { "valid": true, "error": null, "arg_count": 2, "parsed": [ { "required": false, "no_leading_space": false, "nullable": false, "kind": { "type": "content", "mode": "math" }, "form": { "type": "standard" } }, { "required": true, "no_leading_space": false, "nullable": false, "kind": { "type": "content", "mode": "math" }, "form": { "type": "standard" } } ] } ``` **JavaScript** ```ts import { validateArgspec } from "texform"; console.log(JSON.stringify(validateArgspec("o m"), null, 2)); ``` ```text { "argCount": 2, "error": null, "parsed": [ { "form": { "type": "standard" }, "kind": { "mode": "math", "type": "content" }, "noLeadingSpace": false, "nullable": false, "required": false }, { "form": { "type": "standard" }, "kind": { "mode": "math", "type": "content" }, "noLeadingSpace": false, "nullable": false, "required": true } ], "valid": true } ``` An invalid string reports where parsing failed: **Python** ```python import json import texform print(json.dumps(texform.validate_argspec("m{x}"), indent=2)) ``` ```text { "valid": false, "error": "invalid argspec (validate_argspec) at char 2: `m` only supports required braced group syntax `m{}`", "arg_count": null, "parsed": null } ``` **JavaScript** ```ts import { validateArgspec } from "texform"; console.log(JSON.stringify(validateArgspec("m{x}"), null, 2)); ``` ```text { "argCount": null, "error": "invalid argspec (validate_argspec) at char 2: `m` only supports required braced group syntax `m{}`", "parsed": null, "valid": false } ``` ## See also [#see-also] * [`validate_argspec` (Python)](https://texform.dev/api/python/validate-argspec) and [`validateArgspec` (JavaScript)](https://texform.dev/api/js/validate-argspec) — the validation API and result shape. * [Inject custom commands](https://texform.dev/docs/inject-custom-commands) — write an argspec and teach it to the parser. * [Argspec design](https://texform.dev/docs/argspec-design) — why this notation, what it extends, and its known limits. --- # Transforms How TeXForm normalizes a formula: pick a profile, run the phase loop to a fixed point, read the report, and switch phases off per run. https://texform.dev/docs/transforms Normalizing a LaTeX formula has no single correct answer. The form that is ideal for comparing two formulas for equivalence is wrong for an author who wants their shorthand preserved, and the form that strips layout hints for a training corpus would damage author-facing output. So TeXForm never imposes one canonical form. You pick a **profile** that names your downstream use, and a curated rule pipeline produces the canonical form for that use. The transform engine is not a find-and-replace pass. It is a profile selection on top of a multi-phase pipeline driven by a declarative rule set, where every rule declares what it normalizes and what it guarantees. This page follows the four things you actually do with it: pick a profile, understand the pipeline it drives, request its diagnostic report, and switch individual phases off for one run. ## Pick a profile [#pick-a-profile] A `TransformEngine` is built around one profile and normalizes formulas into that profile's canonical form. **Python** ```python import texform engine = texform.TransformEngine(profile="corpus") print(engine.normalize(r"a \over b")) ``` ```text \frac { a } { b } ``` **JavaScript** ```js import { TransformEngine } from 'texform'; const engine = new TransformEngine({ profile: 'corpus' }); console.log(engine.normalize('a \\over b')); ``` ```text \frac { a } { b } ``` `normalize` returns the normalized string. The diagnostic report is a separate call, described in Reading the report below. Build an engine once per profile and reuse it across many formulas; the rule plan is compiled when the engine is constructed. Each profile targets one downstream scenario: | Profile | Target scenario | | ----------- | ----------------------------------------------------------------------------------------------------- | | `Authoring` | Polished author-facing output; stylistic choices kept. | | `Faithful` | Same rendered formula; convenience macros expanded into universal forms. | | `Corpus` | Complete canonical forms suitable as training labels; reading-identical layout variants may collapse. | | `Equiv` | Aggressive intermediates for equivalence comparison, deduplication, or fingerprints. | The profiles form a ladder. Each one applies every rule the profile above it applies and then normalizes more aggressively, trading away progressively more of the source's surface form. `Authoring` keeps the author's intent; `Faithful` expands convenience macros while preserving rendering; `Corpus` additionally collapses reading-identical forms into complete training labels; `Equiv` adds rules that may discard visually salient notation choices. The same input under all four profiles makes the ladder concrete (real engine output, with `physics` loaded so `\dv` is known): | Input | `Authoring` | `Faithful` | `Corpus` | `Equiv` | | ------------------------------------- | --------------------------------------- | ----------------------------------------------- | ------------------------------------------------------ | ------------------------------------------------------ | | `a \over b` | `\frac { a } { b }` | `\frac { a } { b }` | `\frac { a } { b }` | `\frac { a } { b }` | | `\dv{f}{x}` | `\dv { f } { x }` | `\frac { \mathrm { d } f } { \mathrm { d } x }` | `\frac { \mathrm { d } f } { \mathrm { d } x }` | `\frac { \mathrm { d } f } { \mathrm { d } x }` | | `\cos{A}` | `\cos { A }` | `\cos { A }` | `\cos A` | `\cos A` | | `\begin{pmatrix} a & b \end{pmatrix}` | `\begin {pmatrix} a & b \end {pmatrix}` | `\begin {pmatrix} a & b \end {pmatrix}` | `\left ( \begin {matrix} a & b \end {matrix} \right )` | `\left ( \begin {matrix} a & b \end {matrix} \right )` | | `\cfrac{a}{b}` | `\cfrac { a } { b }` | `\cfrac { a } { b }` | `\cfrac { a } { b }` | `\frac { a } { b }` | | `\widehat{x}` | `\widehat { x }` | `\widehat { x }` | `\widehat { x }` | `\hat { x }` | Each row isolates one rung of the ladder: * `a \over b` modernizes legacy infix syntax into `\frac` under **every** profile. This is uncontroversial standardization that no scenario wants to keep. * `\dv{f}{x}` is a `physics` convenience macro. `Authoring` keeps the shorthand; `Faithful` and stronger profiles expand it into the universal `\frac{\mathrm{d} f}{\mathrm{d} x}` form that renders without the `physics` package. * `\cos{A}` keeps its redundant braces under `Authoring` and `Faithful`, which preserve spacing context, but drops them under `Corpus` and `Equiv`, which strip spacing hints a corpus should not learn. * `\begin{pmatrix}` enters at `Corpus`, which rewrites the fenced matrix into explicit `\left(` / `\right)` delimiters around a bare `matrix`, so reading-identical specialized spellings collapse to one form. * `\cfrac{a}{b}` and `\widehat{x}` enter at `Equiv`. Both rewrites lose something a reader can see — the centered continued-fraction layout, the accent stretched to its base's width — so `Corpus` keeps them as valid labels while `Equiv` folds them into `\frac` and `\hat`. To choose: cleaning a training corpus selects `Corpus`; comparing formulas for equivalence selects `Equiv` (paired with a CAS for true mathematical equivalence, covered below); keeping the rendered formula unchanged while expanding macros selects `Faithful`; producing author-facing output selects `Authoring`. The decision is downstream-driven, not a quality dial — a "stronger" profile is not a "better" profile. `Equiv` output is the wrong training label for the image it came from, because comparison-only rules need not preserve notation; `Authoring` output is the wrong input for an equivalence check, because equivalent formulas can still differ as strings. Pick the profile whose target scenario matches what you will do with the output. ## The pipeline [#the-pipeline] Behind a profile, normalization runs as an ordered sequence of phases. The profile selects which rules are active; the phase order itself is fixed. Normalization repeats LowerAttributes → Rewrite → LowerAttributes → FinalizeAst → FlattenGroups until all enabled phases reach a common fixed point. A read-only eliminated-form check follows. Each round follows LowerAttributes → Rewrite → LowerAttributes → FinalizeAst → FlattenGroups. Each phase reaches its own fixed point; any mutation invalidates the other phases. A phase that has already processed the current tree is skipped. Rounds continue until all enabled phases agree on a fixed point, with a bounded round limit that returns a convergence error rather than looping forever. The eliminated-form check runs once afterward: it reads the final tree and changes nothing. The following sections describe each distinct phase with a verified before/after example. ### Rewrite [#rewrite] Rewrite applies the profile's rule set in a fixed-point loop: it keeps applying rules until an iteration changes nothing or the iteration cap is reached. Each rule is declarative — it states the forms it watches for, the forms it eliminates, and the forms it produces — so the engine schedules rules rather than running a fixed script. This phase is where legacy syntax is modernized and convenience macros are expanded. `a \over b` becomes `\frac { a } { b }`; `\dv{f}{x}` becomes `\frac { \mathrm { d } f } { \mathrm { d } x }` under `Faithful` and stronger. Which rules are active is decided by the profile's rule levels, described under [What the engine guarantees](#what-the-engine-guarantees). The fixed-point loop matters when one rule's output is another rule's input. Expanding `\dv{f}{x}` produces a `\frac` whose arguments contain a `\mathrm` wrapper; a later iteration can then normalize that wrapper further. The engine keeps iterating until a full pass changes nothing, which is why the report for even a single-rule input often shows more than one iteration — the final pass is the one that confirms the tree has stabilized. The loop is bounded by a configurable cap so a misbehaving rule set cannot spin forever. ### LowerAttributes [#lowerattributes] LowerAttributes canonicalizes font and style markup so that the two LaTeX ways of writing the same styling converge to one form. The declarative form `{\bf x}` and the prefix form `\mathbf{x}` both become `\mathbf { x }`: **Python** ```python import texform engine = texform.TransformEngine(profile="corpus") print(engine.normalize(r"{\bf x}")) print(engine.normalize(r"\mathbf{x}")) ``` ```text \mathbf { x } \mathbf { x } ``` **JavaScript** ```js import { TransformEngine } from 'texform'; const engine = new TransformEngine({ profile: 'corpus' }); console.log(engine.normalize('{\\bf x}')); console.log(engine.normalize('\\mathbf{x}')); ``` ```text \mathbf { x } \mathbf { x } ``` The phase tracks declarative scope across container boundaries, so a declarative command's effect is bounded correctly instead of leaking into following siblings. ### FinalizeAst [#finalizeast] FinalizeAst is the profile-neutral cleanup pass: it canonicalizes the tree's shape without consulting any rule metadata. It does two kinds of work. It merges adjacent `Prime` nodes, so a prime written as `\prime` marks and a prime written as quote shorthand reach the same final shape. And it normalizes text sequences — merging adjacent text-mode siblings, collapsing runs of ordinary whitespace to a single space without trimming the edges, and cleaning out empty text. **Python** ```python import texform engine = texform.TransformEngine(profile="corpus") print(engine.normalize(r"f^{\prime\prime}")) print(engine.normalize(r"f''")) print(engine.normalize(r"\text{a{}b}")) ``` ```text f'' f'' \text {ab} ``` **JavaScript** ```js import { TransformEngine } from 'texform'; const engine = new TransformEngine({ profile: 'corpus' }); console.log(engine.normalize('f^{\\prime\\prime}')); console.log(engine.normalize("f''")); console.log(engine.normalize('\\text{a{}b}')); ``` ```text f'' f'' \text {ab} ``` The pass is idempotent: revisiting an already-canonical tree does not count additional mutations. ### FlattenGroups [#flattengroups] FlattenGroups removes structurally redundant braces — empty groups, single-child groups in argument slots, and nested groups that add no structure — once the earlier phases have stabilized. Redundant nesting like `{{x}}` collapses to `x` under every profile: **Python** ```python import texform engine = texform.TransformEngine(profile="corpus") print(engine.normalize(r"{{x}}")) ``` ```text x ``` **JavaScript** ```js import { TransformEngine } from 'texform'; const engine = new TransformEngine({ profile: 'corpus' }); console.log(engine.normalize('{{x}}')); ``` ```text x ``` Flattening is gated so it never changes meaning by accident. The public FlattenGroups switches are `enabled` and `preserve_rendered_spacing`. Structural (semantic) guards stay on whenever the phase runs: they protect script binding, environment cell boundaries, declarative scope, infix scope, the brace groups after an unknown command, and a group whose opening delimiter the command before it would otherwise read as an argument. With `physics` loaded, for example, `\sin{[2]}x` keeps its braces, because `\sin[2]x` would read `2` as the exponent. `preserve_rendered_spacing` turns the spacing guards on or off together: `Authoring` and `Faithful` keep it true; `Corpus` and `Equiv` set it false. That is exactly why `\cos{A}` keeps its braces under `Authoring` but loses them under `Corpus`. ## What the engine guarantees [#what-the-engine-guarantees] Two mechanisms back the claim that normalization is correct rather than best-effort. **Normalization is gated on a complete tree.** A tree with `Error` nodes cannot be normalized — normalizing a formula with holes is meaningless — so the engine refuses it. This is why the transform path begins by ensuring the parse is clean: **Python** ```python import texform engine = texform.TransformEngine(profile="corpus") try: engine.normalize(r"x^") except texform.ParseError as exc: print("rejected:", exc) ``` ```text rejected: parse produced an incomplete document ``` **JavaScript** ```js import { TransformEngine, TexformParseError } from 'texform'; const engine = new TransformEngine({ profile: 'corpus' }); try { engine.normalize('x^'); } catch (exc) { console.log('rejected:', exc instanceof TexformParseError); } ``` ```text rejected: true ``` The same gate applies to `transform`, which normalizes a live `Document` in place: it accepts only complete documents that came from its own engine's parser. **The eliminated-form contract is enforced, not assumed.** Each rewrite rule declares the forms it eliminates — forms that must not appear anywhere in the output. After every mutating phase has run, the engine checks that no eliminated form survives. A remaining eliminated form is a hard transform error surfaced as a contract violation, not a silent warning. The same contract is re-checked continuously against real-world datasets through corpus regression, so a rule that fails to eliminate what it promised is caught against actual formulas, not only unit tests. A rule's **rule level** is what decides whether a profile runs it. `RuleLevel` is ordered `Authoring < Faithful < Corpus < Equiv`, and a rule's level is the first profile that accepts the rule's output as a suitable product: `Authoring` rules run under every profile; `Faithful` rules run under Faithful and up; `Corpus` rules under Corpus and up; `Equiv` rules only under Equiv. Each profile turns on its own level plus every level below it, so moving up the ladder only adds rules. A rule's **fidelity** is a separate worst-case equivalence guarantee, ordered `Math < Reading < Render`. Level answers "which use case first accepts this output"; `RuleFidelity` answers "which equivalence relation holds over the declared input domain". Authoring, Faithful, and Corpus require at least `Reading`, while Equiv permits `Math`. A stronger fidelity does not automatically place a rule in an earlier level. The rule that drops fixed delimiter sizing, turning `\big( x \big)` into `( x )`, declares `Reading` fidelity — notation identity, reading order, and structural roles all survive — and is still an `Equiv` rule, because a training label should keep the size the author chose. You can browse every rule with its level and fidelity in the [rule reference](https://texform.dev/rules). ## Reading the report [#reading-the-report] Plain `normalize` returns the canonical string and does not build a report. Plain `transform` updates a document in place and returns nothing (`None` in Python, `void` in JavaScript). Ask for the diagnostic report with the same source and configuration on a separate method: `normalize_with_report` / `transform_with_report` in Python, and `normalizeWithReport` / `transformWithReport` in JavaScript. `normalize_with_report` and `normalizeWithReport` return a `NormalizeReportResult` with `normalized` and `report`. The transform report methods return the report alone. Output text and errors follow the plain transform contract. Report fields, phase grouping, and counters are diagnostic and are not a stable compatibility promise. The report has one section per phase: `lower_attributes`, `rewrite` (`iterations` and `rules`), `finalize_ast` (`prime_run_merges` and `text_normalizations`), and `flatten_groups` (`actions` and `guard_hits`). `guard_hits` has one counter per preserve guard and structural protection: `unknown_command_arguments`, `argument_capture`, `declarative_scope`, `script_base`, `env_body`, `infix_scope`, `command_contact`, `command_contact_via_scripted_base`, `empty_group`, `lone_atom_spacing_char`, `leading_atom_spacing_char`, and `delimited_pair`. `command_contact_via_scripted_base` is also included in `command_contact`, so the two are not independent totals; do not add them. This input exercises four phases at once — a declarative font command, a legacy infix, two `\prime` commands, and an empty text group: **Python** ```python import texform engine = texform.TransformEngine(profile="corpus") result = engine.normalize_with_report(r"{\bf x} + {a \over b} + f^{\prime\prime} + \text{a{}b}") report = result["report"] print(result["normalized"]) print("iterations:", report["rewrite"]["iterations"]) print("rules:", [(r["key"], r["applied_count"]) for r in report["rewrite"]["rules"]]) print("finalize_ast:", report["finalize_ast"]) print("flatten_groups:", report["flatten_groups"]["actions"]) print("lower_attributes:", report["lower_attributes"]["attributes"]) ``` ```text \mathbf { x } + \frac { a } { b } + f'' + \text {ab} iterations: 4 rules: [('base/over-to-frac', 1), ('base/prime-to-prime-node', 2)] finalize_ast: {'prime_run_merges': 1, 'text_normalizations': 1} flatten_groups: {'removed_empty': 1, 'replaced_single_child': 0, 'inlined_multi_child': 0, 'unwrapped_slot': 2} lower_attributes: [{'attr': 'math_font', 'value': 'bold', 'consumed': {'declaratives': 1, 'prefixes': 3}, 'redundant': {'declaratives': 0, 'prefixes': 0}, 'emitted': {'declaratives': 0, 'prefixes': 4}}] ``` **JavaScript** ```js import { TransformEngine } from 'texform'; const engine = new TransformEngine({ profile: 'corpus' }); const result = engine.normalizeWithReport('{\\bf x} + {a \\over b} + f^{\\prime\\prime} + \\text{a{}b}'); const report = result.report; console.log(result.normalized); console.log('iterations:', report.rewrite.iterations); console.log('rules:', report.rewrite.rules.map((r) => [r.key, r.appliedCount])); console.log('finalizeAst:', report.finalizeAst); console.log('flattenGroups:', report.flattenGroups.actions); console.log('lowerAttributes:', report.lowerAttributes.attributes); ``` Reading it phase by phase: * **`rewrite.iterations`** counts Rewrite iterations across the phase rounds: 4 on this input. The count includes unchanged convergence checks after other phases mutate the tree. * **`rewrite.rules`** lists each rule that recorded activity, with how many times it applied. Here `base/over-to-frac` fired once, turning `a \over b` into `\frac`, and `base/prime-to-prime-node` fired twice, turning each `\prime` command into a `Prime` node. * **`finalize_ast.prime_run_merges`** fired once, fusing the two `Prime` nodes Rewrite had just produced into the `f''` shorthand. **`finalize_ast.text_normalizations`** fired once, removing the empty group inside `\text{a{}b}`. * **`flatten_groups.actions`** records one `removed_empty` and two `unwrapped_slot` removals: the empty `{}` is gone, and the braces wrapping `a \over b` and the `\prime` script were redundant single-child groups in argument and script slots. **`flatten_groups.guard_hits`** counts the first preserve branch reached for a group; every counter is zero on this input. * **`lower_attributes.attributes`** shows the `math_font: bold` unit. The counters accumulate across all phase rounds: the first pass consumes the `{\bf x}` declarative and emits the `\mathbf{x}` prefix form; later visits re-canonicalize that prefix as other phases change the tree. Here `consumed` totals one declarative and three prefixes, while `emitted` totals four prefixes. JavaScript reports the same data with camelCase field names — `finalizeAst`, `primeRunMerges`, `textNormalizations`, `guardHits`, `removedEmpty` — and each rule entry uses `key`, `appliedCount`, `skippedCount`. A disabled phase stays in the report as zeros or an empty list. These counters are not a single "number of edits": do not add them together and read the sum as output changes. ## Switch a phase off with an overlay [#switch-a-phase-off-with-an-overlay] A profile sets sensible defaults, but you can override individual phases for one run by passing an **overlay**: a partial, per-phase set of switches merged onto the profile's own config. Disabling Rewrite leaves legacy syntax untouched; disabling FlattenGroups keeps redundant braces. **Python** ```python import texform engine = texform.TransformEngine(profile="corpus") # Overlays are keyword arguments holding per-phase dicts. print(engine.normalize(r"a \over b", rewrite={"enabled": False})) print(engine.normalize(r"\cos{A}", flatten_groups={"enabled": False})) # Or start from the profile's complete config and change a field. config = texform.TransformConfig.corpus() config.rewrite.max_iterations = 50 print(engine.normalize(r"a \over b", config=config)) ``` ```text a \over b \cos { A } \frac { a } { b } ``` **JavaScript** ```js import { TransformEngine } from 'texform'; const engine = new TransformEngine({ profile: 'corpus' }); // The overlay is a nested, camelCase object; one key per phase. console.log(engine.normalize('a \\over b', { rewrite: { enabled: false } })); console.log(engine.normalize('\\cos{A}', { flattenGroups: { enabled: false } })); ``` ```text a \over b \cos { A } ``` The two languages differ in how they express a *complete* config, and the difference is worth knowing before you copy code between them. In Python, `config=` takes a `TransformConfig` instance whose four phase fields are all required, so you build one with `TransformConfig.authoring()`, `faithful()`, `corpus()`, or `equiv()` and then adjust fields; passing a plain dict as `config` raises a configuration error. In JavaScript there is no complete config class at all — every per-run object is an overlay, and `defaultTransformConfig()` is how you read the effective values. In both languages the overlay is nested one level per phase: `lower_attributes` / `lowerAttributes`, `rewrite`, `finalize_ast` / `finalizeAst`, `flatten_groups` / `flattenGroups`. An unknown key, or a Python-style key handed to the JavaScript binding, raises a configuration error rather than being silently ignored. An overlay changes *which phases run*, never which rules the profile selected. If you want the pipeline intact but one specific rewrite rule out of it, disable that rule when you construct the engine instead — `disable_rules=["physics/dv-to-frac-d"]` in Python, `disableRules` in JavaScript. The full set of switches is documented on the [transform engine API page](https://texform.dev/api/python/transform-engine). ## What the engine does not do [#what-the-engine-does-not-do] The transform engine normalizes the surface form of a formula. It does not perform mathematical reasoning: it will not prove that `a + b` equals `b + a`, evaluate an integral, or simplify an algebraic expression. The `Equiv` profile narrows the gap by canonicalizing equivalent spellings to the same string, so a string comparison catches many cases for free — but true mathematical equivalence needs a computer algebra system. Treat `Equiv` as a cheap first pass that resolves spelling differences, and reserve a CAS for the cases that survive it. The engine also does not render: it produces LaTeX, not pixels. Fidelity levels are metadata contracts the engine declares, not comparisons it runs; verifying that two formulas render identically is the job of a separate renderer-validation step, not of normalization. - [Glossary](https://texform.dev/docs/glossary) — Profile, rule level, fidelity, overlay, phase — the vocabulary on this page, defined. - [Serialization](https://texform.dev/docs/serialization) — The output channels and text idempotency behind normalization. - [Normalize a corpus](https://texform.dev/docs/normalize-a-corpus) — Run the engine over a dataset, with error handling and statistics. - [Compare formulas](https://texform.dev/docs/compare-formulas) — Use the equiv profile with a CAS for equivalence checking. --- # Serialization How TeXForm turns a Document back into LaTeX text, tokens, or a structured snapshot, with a text idempotency guarantee and configurable style. https://texform.dev/docs/serialization Once you have a `Document`, you turn it back into something usable: LaTeX text to store or display, tokens to align with that text, or a structured snapshot to transport. TeXForm names each channel for what it produces and guarantees that the text channel is stable under re-parsing, which is what makes its output safe to cache, deduplicate, and diff. ## What serialization is [#what-serialization-is] The two channels you reach for most are named for what they produce rather than a generic "serialize": **Python** ```python import json import texform doc = texform.Parser().parse(r"\frac{x}{y}")["document"] print(doc.to_latex()) # text channel print(json.dumps(doc.to_syntax(), indent=2)) # structured channel ``` ```text \frac { x } { y } { "Root": { "mode": "Math", "children": [ { "Command": { "name": "frac", "args": [ { "kind": "Mandatory", "no_leading_space": false, "value": { "MathContent": { "Char": "x" } } }, { "kind": "Mandatory", "no_leading_space": false, "value": { "MathContent": { "Char": "y" } } } ], "known": true } } ] } } ``` **JavaScript** ```js import { Parser } from 'texform'; const doc = new Parser().parse('\\frac{x}{y}').document; console.log(doc.toLatex()); // text channel console.log(JSON.stringify(doc.toSyntax(), null, 2)); // structured channel ``` ```text \frac { x } { y } { "Root": { "mode": "Math", "children": [ { "Command": { "name": "frac", "args": [ { "kind": "Mandatory", "no_leading_space": false, "value": { "MathContent": { "Char": "x" } } }, { "kind": "Mandatory", "no_leading_space": false, "value": { "MathContent": { "Char": "y" } } } ], "known": true } } ] } } ``` ## Channels, named on purpose [#channels-named-on-purpose] The channels answer different questions, and the naming keeps them apart: * **`to_latex(options?)`** renders the tree back to LaTeX *text* in the selected style. This is the human-readable, re-parseable output. There is intentionally no method named `serialize` on `Document` — a generic name would hide which channel you meant. * **`to_tokenized_latex(options?)`** produces that same text plus a typed token per output fragment, from one traversal. See [Tokenized output](#tokenized-output). * **`to_syntax()`** converts the tree to a `SyntaxNode`, the single structured-data format. Structured output — JSON snapshots, Python dictionaries, JavaScript objects, test fixtures — always goes through `SyntaxNode`. `Document` itself does not serialize to structured data directly. Reach for `to_latex()` when you need LaTeX a renderer or another tool can read; reach for `to_tokenized_latex()` when a downstream consumer needs to point back into that string; reach for `to_syntax()` when you need to store or transport the tree and rebuild it later with `Document.from_syntax()`. A module-level `serialize` function also exists in both bindings, taking a `SyntaxNode` instead of a `Document`. In JavaScript it is marked `@deprecated`: prefer `Document.fromSyntax(node).toLatex(options)`, or `document.toLatex(options)` when you already hold a document. ## Text idempotency [#text-idempotency] Both output styles guarantee **text idempotency**: parsing the output and serializing it again in the same style always produces the same string. Formally: ```text serialize(parse(serialize(parse(src)))) == serialize(parse(src)) ``` In practice, serializing a `Document` and then parsing-and-serializing that output again yields a fixed point: **Python** ```python import texform parser = texform.Parser() once = parser.parse(r"\frac{x}{y}")["document"].to_latex() twice = parser.parse(once)["document"].to_latex() print(once == twice, repr(once)) ``` ```text True '\\frac { x } { y }' ``` **JavaScript** ```js import { Parser } from 'texform'; const parser = new Parser(); const once = parser.parse('\\frac{x}{y}').document.toLatex(); const twice = parser.parse(once).document.toLatex(); console.log(once === twice, JSON.stringify(once)); ``` ```text true "\\frac { x } { y }" ``` Two things this guarantee is, and is not: * It is a **text-level** guarantee. The string is stable; the AST is not required to be identical across the round-trip. `parse(serialize(ast))` may recover a different but equivalent tree kind. * It is **not normalization**. Serialization does not rewrite legacy syntax — `a \over b` serializes back as `a \over b`, not `\frac{a}{b}`. Canonicalizing the formula's content is the transform engine's job, covered in [Transforms](https://texform.dev/docs/transforms). Text idempotency is what makes serialized output safe to use as a key: because the serialized text is stable, you can cache by it, deduplicate identical formulas, and produce diffs that reflect real changes rather than spelling noise. ## The default style is spaced [#the-default-style-is-spaced] By default the serializer writes the **spaced** style: in math mode, tokens, braces, and scripts are separated by single spaces, so `\frac{x}{y}+x_i^2` becomes `\frac { x } { y } + x _ { i } ^ { 2 }`. A space at every token boundary means a downstream tokenizer never has to decide whether `}{` is one token or two, and the spaces are insignificant to LaTeX renderers, so the spaced form renders identically to a compact spelling. Adjacent digits stay together, text-mode content keeps its own spacing, and inline math binds tightly to `$`. Spaced output is byte-stable across releases, so datasets, caches, and deduplication keys built on it stay valid when you upgrade. ## Output styles [#output-styles] `to_latex()` takes one serializer option, `style`, with two values: | Style | Purpose | Example | | ------------------ | ------------------------------------ | --------------------------------------- | | `spaced` (default) | Stable output for machine processing | `\frac { a } { b } + x _ { i } ^ { 2 }` | | `compact` | Concise output for people and LLMs | `\frac{a}{b}+x_i^2` | Compact keeps required lexical separators, such as `\alpha x` and a row break followed by sibling brackets (`\\ [6pt]`). It omits script braces only for a single TeX token: `x_i`, `x^2`, and `x_\alpha`, but `x^{10}` and `x^{\frac{a}{b}}` keep their braces. A following ASCII letter or digit receives a readability space (`x^2 3`, `x_i y`). Both styles put subscripts before superscripts and brace infix operands only when needed. Both styles preserve the parsed structure: parsing either output gives back the same tree. Compact layout may still improve across releases, so prefer spaced output when you store results and compare them later. The style changes only spacing and optional braces; it does not choose a transform profile or rewrite commands. **Python** ```python import texform doc = texform.Parser().parse(r"x_i^2")["document"] print(doc.to_latex()) # x _ { i } ^ { 2 } print(doc.to_latex(style="compact")) # x_i^2 ``` **JavaScript** ```js import { Parser } from 'texform'; const doc = new Parser().parse('x_i^2').document; console.log(doc.toLatex()); // x _ { i } ^ { 2 } console.log(doc.toLatex({ style: 'compact' })); // x_i^2 ``` Omitting `style`, or passing Python `None` or JavaScript `null`, selects spaced output. An unknown style or option key raises a configuration error instead of falling back to the default. For normalized output and the command line, see [Configure output style](https://texform.dev/docs/configure-output-style); for the option reference, see the [serialize API page](https://texform.dev/api/python/serialize). ## Tokenized output [#tokenized-output] Sometimes the LaTeX string is not enough: a viewer needs to highlight a subexpression, an aligner needs to map model tokens onto source fragments, a diff needs to know which fragment changed. `to_tokenized_latex()` answers that by returning the serialized string together with one typed token per output fragment. It is the same traversal, not a second pass. The tokens are recorded while the serializer walks the tree, so `latex` is byte-for-byte what `to_latex()` would have produced with the same `style`, and the token spans follow that style's spacing. **Python** ```python import texform doc = texform.Parser().parse(r"\text{é} + x")["document"] tok = doc.to_tokenized_latex() print(repr(tok["latex"])) for t in tok["tokens"]: print(t["kind"], t["mode"], t["start_byte"], t["end_byte"], repr(t["text"])) ``` ```text '\\text {é} + x' control_sequence math 0 5 '\\text' delimiter math 6 7 '{' text text 7 9 'é' delimiter math 9 10 '}' character math 11 12 '+' character math 13 14 'x' ``` **JavaScript** ```js import { Parser } from 'texform'; const doc = new Parser().parse('\\text{é} + x').document; const tok = doc.toTokenizedLatex(); console.log(JSON.stringify(tok.latex)); for (const t of tok.tokens) { console.log(t.kind, t.mode, t.startByte, t.endByte, JSON.stringify(t.text)); } ``` Three things to read out of that output. **Each token carries a kind and a mode.** The kind is one of `control_sequence`, `character`, `delimiter`, `text`, `raw`, or `error` — semantic categories the serializer owns, not lexer categories. The mode is the fragment's semantic math or text mode, independent of spacing: `é` is `text` because it sits inside a `\text` argument, while the braces around it are math-mode delimiters belonging to the surrounding expression. > **Warning** > > `start_byte` and `end_byte` are **UTF-8 byte offsets** into `latex`, not Python code-point indices and not JavaScript UTF-16 indices. The final `x` above spans bytes 13–14 of a string that is only 13 characters long, because `é` occupies two bytes. Slice the encoded bytes — `tok["latex"].encode()[13:14]` in Python, `Buffer.from(tok.latex, 'utf8').subarray(13, 14)` or a `TextEncoder` view in JavaScript — rather than indexing the string directly. **Spans are ordered and never overlap.** The gaps between them are the serializer's own spacing, so the tokens are the content of the string and the spaces between tokens are layout. An `Error` node contributes an `error` token holding its captured snippet, and an *empty* error snippet contributes no token at all rather than a zero-width one — so a token list can be shorter than you expect on a partial tree. Use `has_errors()` to ask whether the document contains error nodes; do not infer it from the token list. ## Boundaries of the guarantee [#boundaries-of-the-guarantee] Two node kinds show how the serializer handles content that is not ordinary math. **Error nodes round-trip verbatim.** A partial tree containing `Error` nodes still serializes losslessly: each error re-emits the exact source snippet it captured, so nothing is dropped. **Python** ```python import texform doc = texform.Parser().parse(r"x^")["document"] print(doc.has_errors(), repr(doc.to_latex())) ``` ```text True 'x^' ``` **JavaScript** ```js import { Parser } from 'texform'; const doc = new Parser().parse('x^').document; console.log(doc.hasErrors(), JSON.stringify(doc.toLatex())); ``` ```text true "x^" ``` **Primes serialize compactly when pure.** A pure prime superscript is written back as quote shorthand — `f''` rather than `f^{\prime\prime}`. A mixed superscript that combines a prime with other content, such as `f'^2`, stays an ordinary script group (`f ^ { ' 2 }`), because compact shorthand cannot express the mix. - [Pipeline](https://texform.dev/docs/pipeline) — Where each output channel sits in the overall flow. - [Transforms](https://texform.dev/docs/transforms) — Normalize a formula's content before serializing it. - [Configure output style](https://texform.dev/docs/configure-output-style) — Choose spaced or compact output in libraries, normalization, and the CLI. - [Glossary](https://texform.dev/docs/glossary) — Text idempotency, overlay, and the rest of the vocabulary. --- # Glossary TeXForm terminology in one place: profile, rule level, fidelity, SyntaxNode, Document, parse config, overlay, phase, and the contracts behind them. https://texform.dev/docs/glossary TeXForm's vocabulary is small but precise, and several terms look interchangeable until you need the difference — profile versus rule level, `SyntaxNode` versus `Document`, config versus overlay. Each entry below says what the term means and links to the page that defines it in context. ## Trees and documents [#trees-and-documents] ### `SyntaxNode` [#syntaxnode] The lossless, immutable snapshot of a parse, and the single structured-data format in TeXForm. It is what you serialize to JSON, store, transport, or use as a test fixture; it carries no editing behavior. Python sees it as a plain dict, JavaScript as a plain object, Rust as a serde type. It may contain `Error` nodes, because a partial parse is still a legitimate snapshot. See [Pipeline](https://texform.dev/docs/pipeline#syntaxnode-is-the-wire-format-document-is-the-working-format). ### `Document` [#document] The editable working tree, and the type nearly every API takes or returns. You query it with node handles, mutate it with validated edits, serialize it with `to_latex()`, and normalize it with the transform engine. A `Document` whose `has_errors()` is true is read-only, fixed that way at construction. See [Parsing](https://texform.dev/docs/parsing#what-parsing-gives-you) and [Edit a document](https://texform.dev/docs/edit-a-document). ### `Ast` [#ast] The internal arena tree the serializer and the transform engine operate on. `Document` wraps it and exposes a fallible surface over it, which is why no internal panic reaches a caller. `Ast` is **not** part of the public API and is not reachable from Python or JavaScript at all; the name appears in phase names like FinalizeAst and in [Architecture](https://texform.dev/docs/architecture), nowhere else. You never construct or hold one. ### Text idempotency [#text-idempotency] The serializer's formal guarantee that parsing its output and serializing it again in the same style produces the same string: `serialize(parse(serialize(parse(src)))) == serialize(parse(src))`. It is a text-level guarantee — the recovered tree need not be the identical tree kind — and it is what makes serialized output safe to use as a cache key, a deduplication key, or a diff input. It is not normalization: serialization never rewrites `a \over b` into `\frac{a}{b}`. See [Serialization](https://texform.dev/docs/serialization#text-idempotency). ## Parsing [#parsing] ### `ParseConfig`, lenient and strict [#parseconfig-lenient-and-strict] The two knobs that decide how the parser reacts to trouble: `reject_unknown` for names the knowledge base does not recognize, and `abort_on_error` for syntax errors it cannot continue past. For both, `True` means stricter. **Lenient** is the corner where both are `False` — the default in every binding — and **strict** is the corner where both are `True`. Rust exposes those corners as the constants `ParseConfig::LENIENT` and `ParseConfig::STRICT`; Python and JavaScript express them through constructor defaults and per-call overlays instead, so there is no `ParseConfig.LENIENT` constant to import. See [Parsing](https://texform.dev/docs/parsing#configuring-strictness). ### Context item [#context-item] A record you hand the parser to inject knowledge it does not have: a command, an environment, or a delimiter control, each with a name and an argspec. The parser then reads the item's arguments exactly as if it were built in. Context items add *parsing* knowledge only — they cannot add rewrite rules. See [Inject custom commands](https://texform.dev/docs/inject-custom-commands). ## The transform engine [#the-transform-engine] ### Profile [#profile] Your choice of downstream use, and the only decision the engine requires: `Authoring`, `Faithful`, `Corpus`, or `Equiv`. A profile is fixed when you construct a `TransformEngine`, and it determines both which rules are active and the default per-phase switches. Profiles form a ladder rather than a quality dial — a stronger profile is not a better one, just a different target. See [Transforms](https://texform.dev/docs/transforms#pick-a-profile). ### `RuleLevel` [#rulelevel] The level every rewrite rule belongs to, ordered `Authoring < Faithful < Corpus < Equiv`. A rule's level is the first profile that accepts the rule's output as a suitable product, and each profile runs its own level plus every level below it. This is what "a profile selects rules" means mechanically. See [What the engine guarantees](https://texform.dev/docs/transforms#what-the-engine-guarantees). ### `RuleFidelity` [#rulefidelity] A rule's worst-case equivalence guarantee over its declared input domain, ordered `Math < Reading < Render`: `Render` means the rendering is equivalent, `Reading` means notation content, reading order, and structural roles survive, and `Math` means only mathematical meaning does. Fidelity is independent of level — a high-fidelity rule is not automatically an earlier-level rule — and it is a metadata contract, not a comparison the engine runs. See [Rules and the transform contract](https://texform.dev/docs/rule-contract). ### `RuleLevelSet` [#rulelevelset] The set of rule levels a compiled engine has turned on, derived from the profile. It is the Rust-side representation of "which rules this engine will attempt". You do not set it directly; choosing a profile chooses it. ### Phase [#phase] One stage of the transform pipeline: LowerAttributes, Rewrite, FinalizeAst, or FlattenGroups. The engine repeats LowerAttributes → Rewrite → LowerAttributes → FinalizeAst → FlattenGroups until all enabled phases reach a common fixed point. You can switch a phase off for a single run. Switching a phase off is not the same as changing rule levels: the profile's rule selection is compiled into the engine and a per-run switch cannot change it. See [Transforms](https://texform.dev/docs/transforms#the-pipeline). ### Overlay [#overlay] A partial configuration merged onto a complete one. Every per-run config in the Python and JavaScript bindings is an overlay: you name only what you want to change, nested one level per phase — `rewrite={"enabled": False}` in Python, `{ rewrite: { enabled: false } }` in JavaScript — and everything you omit keeps the profile's value. An unknown key, or a Python-style key handed to the JavaScript binding, raises a configuration error rather than being ignored. See [Transforms](https://texform.dev/docs/transforms#switch-a-phase-off-with-an-overlay). ### Eliminated-form contract [#eliminated-form-contract] Each rewrite rule declares the forms it eliminates — forms that must not appear anywhere in the engine's output. After every mutating phase has run, the engine checks the final tree and reports a surviving eliminated form as a hard transform error rather than a warning. The same contract is re-checked against real-world datasets through corpus regression, so a rule that fails to eliminate what it promised is caught on actual formulas. See [Rules and the transform contract](https://texform.dev/docs/rule-contract). - [What TeXForm is](https://texform.dev/docs/what-is-texform) — The product boundary, in one page. - [Transforms](https://texform.dev/docs/transforms) — Where profile, rule level, phase, and overlay all come together. - [Rule reference](https://texform.dev/rules) — Every built-in rewrite rule with its level, fidelity, and eliminated forms. --- # Normalize a corpus Run one reused transform engine over a dataset, skipping bad formulas and aggregating rule statistics. https://texform.dev/docs/normalize-a-corpus Normalizing a corpus of LaTeX formulas is a batch job, and two facts shape the loop. First, building a `TransformEngine` compiles its rule plan, so you construct it once and reuse it across every formula rather than per item. Second, real corpora contain formulas the engine cannot normalize: an input that cannot produce a complete tree — a truncated `x^`, an unbalanced `\frac{a}{b` — makes `normalize` raise `ParseError`, which you catch and bucket instead of letting it stop the run. The pattern below builds the engine once, normalizes each formula under the `corpus` profile, sends parse failures to a skip bucket, and tallies which rules fired across the batch so you can see what the normalization actually changed. One knowledge-base default is worth knowing before you point this at real data. The default knowledge base loads the default runtime set — `base`, `ams`, `braket`, `textmacros`, `bboldx`, `boldsymbol`, and `color` — rather than the whole catalog. The catalog also includes the opt-in `physics` package. Without it, physics-only commands such as `\dv` are unknown and pass through unexpanded, and `\ket` keeps its `braket` meaning. The batch below contains a `physics` derivative, so its knowledge base names its packages; see [Troubleshooting](https://texform.dev/docs/troubleshooting) for how the two readings of `\ket` differ. **Python** ```python import texform from collections import Counter formulas = [ r"a \over b", r"\dv{f}{x}", r"{\bf x} + {a \over b}", r"\cos{A}", r"x^", # truncated: cannot produce a complete tree r"\frac{a}{b", # unbalanced: parse returns no document ] kb = texform.KnowledgeBase(packages=["base", "ams", "physics"]) engine = texform.TransformEngine(profile="corpus", knowledge_base=kb) # build once, reuse normalized = [] skipped = [] rule_counts = Counter() for src in formulas: try: result = engine.normalize_with_report(src) except texform.ParseError: skipped.append(src) continue normalized.append(result["normalized"]) for rule in result["report"]["rewrite"]["rules"]: if rule["applied_count"] > 0: rule_counts[rule["key"]] += rule["applied_count"] print(f"normalized {len(normalized)} / {len(formulas)} formulas") print(f"skipped {len(skipped)}: {skipped}") for line in normalized: print(line) for key, count in rule_counts.most_common(): print(f"{key}: {count}") ``` **JavaScript** ```js import { KnowledgeBase, TexformParseError, TransformEngine } from 'texform'; const formulas = [ 'a \\over b', '\\dv{f}{x}', '{\\bf x} + {a \\over b}', '\\cos{A}', 'x^', // truncated: cannot produce a complete tree '\\frac{a}{b', // unbalanced: parse returns no document ]; const knowledgeBase = new KnowledgeBase({ packages: ['base', 'ams', 'physics'] }); const engine = new TransformEngine({ profile: 'corpus', knowledgeBase }); // build once, reuse const normalized = []; const skipped = []; const ruleCounts = new Map(); for (const src of formulas) { let result; try { result = engine.normalizeWithReport(src); } catch (err) { if (err instanceof TexformParseError) { skipped.push(src); continue; } throw err; } normalized.push(result.normalized); for (const rule of result.report.rewrite.rules) { if (rule.appliedCount > 0) { ruleCounts.set(rule.key, (ruleCounts.get(rule.key) ?? 0) + rule.appliedCount); } } } console.log(`normalized ${normalized.length} / ${formulas.length} formulas`); console.log(`skipped ${skipped.length}`); for (const line of normalized) console.log(line); for (const [key, count] of [...ruleCounts].sort((a, b) => b[1] - a[1])) { console.log(`${key}: ${count}`); } ``` The two malformed inputs land in the skip bucket, the four survivors are normalized, and the rule tally shows exactly which rewrite rules carried the batch: ```text normalized 4 / 6 formulas skipped 2: ['x^', '\\frac{a}{b'] \frac { a } { b } \frac { \mathrm { d } f } { \mathrm { d } x } \mathbf { x } + \frac { a } { b } \cos A base/over-to-frac: 2 physics/dv-to-frac-d: 1 ``` Catching `ParseError` only tells you a formula could not be normalized, not why; to split incomplete trees from input that produced no tree at all and decide which to repair, read [Clean OCR/MER output](https://texform.dev/docs/clean-ocr-mer-output), and for the meaning of each rule entry and report field see [Transforms](https://texform.dev/docs/transforms). ## Profile the corpus before you normalize it [#profile-the-corpus-before-you-normalize-it] The rule tally tells you what normalization changed. A different question — what is actually *in* the corpus — is often the one you want first, when you are deciding which packages to load or whether a profile is even appropriate for the data. `count_targets` answers it per formula without building a tree you have to walk: it returns a dict of prefixed keys, `cmd:` for commands, `env:` for environments, and `char:` for named character commands. > **Python only** > > `count_targets` is a Python-only corpus helper. There is no JavaScript equivalent; in JavaScript, walk the parsed document with `findCommands` and `descendants` instead. ```python import texform from collections import Counter corpus = [ r"\frac{x}{y} + \alpha", r"\dv{f}{x}", r"\begin{pmatrix} a & b \end{pmatrix}", r"\alpha + \beta", ] totals = Counter() for src in corpus: totals.update(texform.count_targets(src)) for key, count in sorted(totals.items(), key=lambda kv: (-kv[1], kv[0])): print(f"{key}: {count}") ``` ```text char:alpha: 2 char:beta: 1 cmd:dv: 1 cmd:frac: 1 env:pmatrix: 1 ``` Like `normalize`, it needs a complete parse and raises `ParseError` otherwise, so the same skip bucket works here. Counting `physics` commands such as `dv` across a sample is a quick way to confirm a corpus really needs that package loaded before you commit to a normalization run over all of it. ## Next steps [#next-steps] - [Clean OCR/MER output](https://texform.dev/docs/clean-ocr-mer-output) — Sort noisy recognition output into repairable and discardable buckets. - [Transforms](https://texform.dev/docs/transforms) — What each profile targets and how to read the transform report. - [Disable a rewrite rule](https://texform.dev/docs/disable-a-rule) — Turn off one rule by key when a profile normalizes more than you want. - [Troubleshooting](https://texform.dev/docs/troubleshooting) — Config errors, package conflicts, and the other failures a batch run hits. --- # Clean OCR/MER output Classify math-recognition output by diagnostics into repairable and discardable buckets, then normalize the survivors. https://texform.dev/docs/clean-ocr-mer-output Output from an OCR or math-expression-recognition (MER) model is noisy in predictable ways: it invents commands that do not exist, truncates structures mid-script, and drops closing braces. You cannot normalize all of it, but you should not throw all of it away either — much of it is clean, and some of it is a repairable near-miss. The job is to sort each formula into a bucket before you spend effort on it. The key tool is the parse result itself. Parse with `reject_unknown` enabled so that unrecognized commands surface as diagnostics instead of slipping through silently, then read the three honest parse states. A `None` document means there was no tree to recover and the formula is a discard. A tree where `has_errors()` is true is a partial, read-only tree: the unparseable parts survived as `Error` nodes, the diagnostics tell you what they were, and this is your repair queue. A clean tree is a survivor you normalize immediately. **Python** ```python import texform batch = [ r"\frac{a}{b}", # clean r"a \over b", # legacy syntax, still a clean tree r"\unknowncmd x", # invented command -> Error node, tree preserved r"x^", # truncated superscript -> Error node, tree preserved r"\frac{a}{b", # unbalanced braces -> no tree at all ] parser = texform.Parser() engine = texform.TransformEngine(profile="corpus") config = texform.ParseConfig(reject_unknown=True) # surface unknown commands normalized = [] flagged = [] discarded = [] for src in batch: result = parser.parse(src, config) doc = result["document"] if doc is None: discarded.append(src) elif doc.has_errors(): kinds = sorted({str(d.get("kind")) for d in result["diagnostics"]}) flagged.append((src, ", ".join(kinds))) else: normalized.append(engine.normalize(src)) print("normalized (clean survivors):") for line in normalized: print(" " + line) print("flagged for repair (partial tree):") for src, kinds in flagged: print(f" {src!r} -> {kinds}") print("discarded (no tree):") for src in discarded: print(f" {src!r}") ``` **JavaScript** ```js import { Parser, TransformEngine } from 'texform'; const batch = [ '\\frac{a}{b}', // clean 'a \\over b', // legacy syntax, still a clean tree '\\unknowncmd x', // invented command -> Error node, tree preserved 'x^', // truncated superscript -> Error node, tree preserved '\\frac{a}{b', // unbalanced braces -> no tree at all ]; const parser = new Parser(); const engine = new TransformEngine({ profile: 'corpus' }); const config = { rejectUnknown: true }; // surface unknown commands const normalized = []; const flagged = []; const discarded = []; for (const src of batch) { const result = parser.parse(src, config); const doc = result.document; if (doc === null) { discarded.push(src); } else if (doc.hasErrors()) { const kinds = [...new Set(result.diagnostics.map((d) => String(d.kind)))].sort(); flagged.push([src, kinds.join(', ')]); } else { normalized.push(engine.normalize(src)); } } console.log('normalized (clean survivors):'); for (const line of normalized) console.log(' ' + line); console.log('flagged for repair (partial tree):'); for (const [src, kinds] of flagged) console.log(` ${JSON.stringify(src)} -> ${kinds}`); console.log('discarded (no tree):'); for (const src of discarded) console.log(' ' + JSON.stringify(src)); ``` The clean formulas are normalized, the recoverable defects are flagged with the diagnostic kind that explains them, and only the truly broken input is discarded: ```text normalized (clean survivors): \frac { a } { b } \frac { a } { b } flagged for repair (partial tree): '\\unknowncmd x' -> unknown-command 'x^' -> raw-expected-found discarded (no tree): '\\frac{a}{b' ``` The `unknown-command` kind names a missing signature you can often fix by registering the command. The truncated `x^` is the other shape: `raw-expected-found` says the parser wanted something specific at that position and did not get it. `expected` names what it wanted — `superscript content` — while `found` is `None`, because the input ended rather than offering the wrong thing. An unclosed environment has its own category, and `expected` names the missing `\end`: ```python import texform parser = texform.Parser() config = texform.ParseConfig(reject_unknown=True) print(parser.parse(r"x^", config)["diagnostics"][0]) print(parser.parse(r"\begin{matrix} a & b", config)["diagnostics"][0]) ``` ```text {'kind': 'raw-expected-found', 'message': 'Missing superscript content', 'span': {'start': 2, 'end': 2}, 'expected': ['superscript content'], 'found': None, 'contexts': []} {'kind': 'unclosed-environment', 'message': 'Environment matrix missing closing \\end{matrix}', 'span': {'start': 20, 'end': 20}, 'expected': ['\\end{matrix}'], 'found': None, 'contexts': []} ``` Bucket on `kind` when it is present and fall back to `message` and `span` when it is not, because `kind` is an optional field that only structured categories populate. An empty span sitting at the end of the source, as in both diagnostics above, is the signature of input that was cut off mid-structure. Treat a missing `kind` as "uncategorized", not as a stable identifier you can match on. > **reject_unknown and has_errors() are independent** > > Neither implies the other. `reject_unknown` is a parser knob that decides whether an unrecognized name becomes a diagnostic or survives as a `known: false` node; `has_errors()` is a property of the tree the parser produced. Turning `reject_unknown` on does not guarantee an error tree, and a tree can have errors with `reject_unknown` off — `x^` above does exactly that. Check the tree when you want to know if it is usable, and the config when you want to know how strict the parse was. To read those diagnostics and their spans in full and decide repair versus discard precisely, see [Handle parse errors](https://texform.dev/docs/handle-parse-errors); for what `reject_unknown` and the three parse states mean, see [Parsing](https://texform.dev/docs/parsing). ## Next steps [#next-steps] - [Handle parse errors](https://texform.dev/docs/handle-parse-errors) — Dispatch on all three parse states and read diagnostics and spans. - [Inject custom commands](https://texform.dev/docs/inject-custom-commands) — Fix unknown-command diagnostics by registering the macro's signature. - [Normalize a corpus](https://texform.dev/docs/normalize-a-corpus) — Run the survivors through one reused engine and tally what changed. - [Troubleshooting](https://texform.dev/docs/troubleshooting) — The other failures a recognition pipeline runs into, with real messages. --- # Handle parse errors Dispatch on the three parse-result states, read diagnostics and spans, and choose strict or lenient parsing. https://texform.dev/docs/handle-parse-errors A parse result has exactly three honest states, and robust code dispatches on all three rather than assuming a document always exists. The document is `None` when the parser had nothing to recover; it is present but `has_errors()` when recovery preserved the unparseable parts as read-only `Error` nodes; and it is present and clean otherwise. Each state carries different information: `None` is explained entirely by the diagnostics list, an error tree exposes the failing spans through its `Error` nodes, and a clean tree is ready to serialize or transform. The function below handles every state in one pass. For the no-tree case it reads the first diagnostic's message and span; for the partial case it reads the span off each `Error` node; for the clean case it serializes the tree back to LaTeX. **Python** ```python import texform def classify(parser, src): result = parser.parse(src) doc = result["document"] if doc is None: # Hard failure: no tree. Diagnostics describe what went wrong. first = result["diagnostics"][0] return f"NO TREE {src!r}: {first['message']} at {first['span']}" if doc.has_errors(): # Partial, read-only tree. Error nodes carry the unparseable spans. spans = [n.span() for n in doc.errors()] return f"PARTIAL {src!r}: {len(doc.errors())} error node(s) at {spans}" # Clean, editable tree. return f"CLEAN {src!r}: {doc.to_latex()}" parser = texform.Parser() for src in [r"\frac{x}{y}", r"x^", r"\frac{a}{b", r""]: print(classify(parser, src)) ``` **JavaScript** ```js import { Parser } from 'texform'; function classify(parser, src) { const result = parser.parse(src); const doc = result.document; if (doc === null) { // Hard failure: no tree. Diagnostics describe what went wrong. const first = result.diagnostics[0]; return `NO TREE ${JSON.stringify(src)}: ${first.message} at ${JSON.stringify(first.span)}`; } if (doc.hasErrors()) { // Partial, read-only tree. Error nodes carry the unparseable spans. const spans = doc.errors().map((n) => n.span()); return `PARTIAL ${JSON.stringify(src)}: ${doc.errors().length} error node(s) at ${JSON.stringify(spans)}`; } // Clean, editable tree. return `CLEAN ${JSON.stringify(src)}: ${doc.toLatex()}`; } const parser = new Parser(); for (const src of ['\\frac{x}{y}', 'x^', '\\frac{a}{b', '']) { console.log(classify(parser, src)); } ``` Each input lands in a distinct branch, and empty input is a clean tree rather than a failure: ```text CLEAN '\\frac{x}{y}': \frac { x } { y } PARTIAL 'x^': 1 error node(s) at [{'start': 0, 'end': 2}] NO TREE '\\frac{a}{b': unclosed brace argument at {'start': 8, 'end': 9} CLEAN '': ``` A diagnostic is a record with a `message`, a `kind` (a stable string such as `unknown-command`, or absent for some structural errors), and a `span` locating it in the source. The `span` on an `Error` node and the `span` on a diagnostic both point at the trouble, so either can drive a highlight in your own tooling. ## Spans are UTF-8 byte offsets [#spans-are-utf-8-byte-offsets] A span is a half-open range of **UTF-8 byte** offsets — not Python code-point indices, and not JavaScript UTF-16 indices. On pure ASCII the distinction never shows up, which is exactly why it bites later. Any non-ASCII character earlier in the formula shifts every following span: ```python import texform src = "\\alpha β x^" result = texform.Parser().parse(src) print("characters:", len(src), "bytes:", len(src.encode())) print("error span:", result["document"].errors()[0].span()) print("slice by byte:", src.encode()[10:12]) ``` ```text characters: 11 bytes: 12 error span: {'start': 10, 'end': 12} slice by byte: b'x^' ``` The `β` occupies two bytes, so the span that covers `x^` reads `10..12` even though those characters start at index 9 of the Python string. Index into the encoded bytes and decode the slice, rather than indexing the string directly. ## Choosing strictness [#choosing-strictness] Two independent knobs decide which states you ever see. Both default to lenient — `reject_unknown` and `abort_on_error` are each `False` unless you say otherwise: | Setting | Result on bad input | When to use | | ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ | | `abort_on_error = False` (default) | Syntax errors are recovered as `Error` nodes, and you triage the result with `has_errors()`. Recovery is best-effort: input the parser cannot recover at all still yields no tree. | Noisy input you want to inspect and partially salvage (OCR/MER output, exploration). | | `abort_on_error = True` | The first syntax error per item stops the parse and produces no recovery `Error` nodes. | Input you expect to be clean and want to reject loudly at the first defect. | | `reject_unknown = False` (default) | Unknown commands survive as plain `known: false` nodes. | Corpora full of project macros you would rather keep than reject. | | `reject_unknown = True` | Unknown commands become diagnostics rather than surviving silently. | Catching invented or misspelled commands. | The knobs are orthogonal, and neither one is equivalent to the tree's `has_errors()` signal. `\frac{a}{b` in the run above returned no tree under the default lenient settings, which is why dispatching on all three states matters more than picking the right strictness. For the full four-quadrant behavior of these two knobs and the parse-state model behind them, see [Parsing](https://texform.dev/docs/parsing); for sorting a real recognition batch into repair and discard buckets with these diagnostics, see [Clean OCR/MER output](https://texform.dev/docs/clean-ocr-mer-output). ## Next steps [#next-steps] - [Clean OCR/MER output](https://texform.dev/docs/clean-ocr-mer-output) — Turn these three states into repair and discard buckets over a real batch. - [Parsing](https://texform.dev/docs/parsing) — The knowledge base, the strictness quadrants, and the parse-state model. - [Edit a document](https://texform.dev/docs/edit-a-document) — What you can do with a clean tree once it survives triage. - [Troubleshooting](https://texform.dev/docs/troubleshooting) — Errors raised outside the parser: config, foreign documents, and edits. --- # Edit a document Traverse a Document, build replacement nodes, and write the edited tree back to LaTeX. https://texform.dev/docs/edit-a-document A tool that rewrites formulas — expanding project shorthands, swapping deprecated commands, normalizing markup — needs to walk a parsed `Document`, find the nodes it cares about, and replace them with new structure. TeXForm gives you that working tree directly: you read through node handles, stage replacement subtrees with `create_*` constructors, attach them, and serialize the result back to LaTeX. Edits are validated as you make them and return a structured error on misuse rather than corrupting the tree, and a tree that came back with parse errors is read-only — the model carried over from [Parsing](https://texform.dev/docs/parsing). The task below expands a project shorthand: every `\R` in the source becomes `\mathbb{R}`. The parser does not know `\R`, so under the default lenient configuration it survives as a plain command node — exactly the kind of node a rewriting tool wants to find and replace. **Python** ```python import texform doc = texform.Parser().parse(r"x \in \R, y \in \R")["document"] print("before:", doc.to_latex()) # Find the shorthand command, then replace each occurrence with a freshly # built \mathbb{R}. A slot takes a detached node or source text. for node in doc.find_commands("R"): letter = doc.create_char("R") # stage a detached child replacement = doc.create_command("mathbb", [letter]) doc.replace_with(node, replacement) print("after :", doc.to_latex()) ``` ```text before: x \in \R , y \in \R after : x \in \mathbb { R } , y \in \mathbb { R } ``` **JavaScript** ```js import { Parser } from 'texform'; const doc = new Parser().parse('x \\in \\R, y \\in \\R').document; console.log('before:', doc.toLatex()); // Find the shorthand command, then replace each occurrence with a freshly // built \mathbb{R}. A slot takes a detached node or source text. for (const node of doc.findCommands('R')) { const letter = doc.createChar('R'); // stage a detached child const replacement = doc.createCommand('mathbb', [letter]); doc.replaceWith(node, replacement); } console.log('after :', doc.toLatex()); ``` ```text before: x \in \R , y \in \R after : x \in \mathbb { R } , y \in \mathbb { R } ``` The workflow is the same for any rewrite: traverse with finders like `find_commands` (or the navigation accessors `children`, `descendants`, `parent`), build detached nodes with the `create_*` constructors, attach them with `replace_with`, `append_child`, `insert_before`, and friends, then read the result with `to_latex()`. Staged nodes must be detached when you attach them — reusing an argument that is still bound to another node fails, so build fresh content rather than re-parenting live nodes. Because edits are fallible, misuse surfaces as a structured error instead of a silent corruption or a crash. The clearest case is a tree that parsed with errors: it is read-only, and every editing method rejects the attempt. **Python** ```python import texform doc = texform.Parser().parse(r"x^")["document"] # incomplete superscript print(doc.has_errors(), doc.is_read_only()) try: doc.append_child(doc.root(), doc.create_char("y")) except texform.EditError as exc: print("EditError:", exc) ``` ```text True True EditError: document is read-only ``` **JavaScript** ```js import { Parser } from 'texform'; const doc = new Parser().parse('x^').document; // incomplete superscript console.log(doc.hasErrors(), doc.isReadOnly()); try { doc.appendChild(doc.root(), doc.createChar('y')); } catch (exc) { console.log('error:', exc.constructor.name, '-', exc.message); } ``` ```text true true error: TexformEditError - document is read-only ``` The other guard worth knowing is node ownership: a node belongs to the document that created it, and an edit that mixes two documents is rejected before it can corrupt either tree. Python raises `EditError` with the message `node belongs to a different document`, JavaScript throws `TexformEditError` for the same misuse, and in both the rejection happens before either tree is touched. Building replacements with the `create_*` constructors on the document you are editing, as above, keeps you clear of it. For the full set of editing methods, the `create_*` constructors, and every `EditError` variant, see the [Document API](https://texform.dev/api/python/document); for the read model these edits build on, see [Parsing](https://texform.dev/docs/parsing). Editing the exponent node changes x^2 to x^3 while retaining the base node; serialization reads the updated tree. ## Next steps [#next-steps] - [Document API](https://texform.dev/api/python/document) — Every finder, create_* constructor, and attachment method on the tree. - [Parsing](https://texform.dev/docs/parsing) — The read model, the three parse states, and why error trees are read-only. - [Configure output style](https://texform.dev/docs/configure-output-style) — Choose spaced or compact LaTeX output. - [Troubleshooting](https://texform.dev/docs/troubleshooting) — Ownership, read-only trees, and the other edit failures, with real messages. --- # Analyze formula structure at scale Export documents as node and argument tables with to_columnar() to count commands, argument forms, and depth across a corpus. https://texform.dev/docs/analyze-structure Counting which commands a corpus uses, how deep its formulas nest, or which optional arguments people actually fill is a bulk question, and walking node handles one formula at a time is the wrong shape for it. `to_columnar()` (`toColumnar()` in JavaScript) exports a document as two flat tables of plain columns: one row per node, one row per argument slot. The columns are ordinary lists, so you can loop over them directly or hand them to Arrow and DataFrame tools without copying a tree across the language boundary for every node. ## What the tables contain [#what-the-tables-contain] The node table holds one row per node in preorder, so a parent always precedes its children, and detached nodes are left out. Row indices are local to the snapshot, and `-1` marks a missing row or index. | Column | Meaning | | ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `kind` | Node kind, such as `Command`, `Char`, `Scripted`, or `Environment`. | | `parent` | Row of the parent node, or `-1` for the root. | | `slot`, `slot_index` | Where the node sits in its parent: `child`, `arg`, `script_base`, `subscript`, `superscript`, `infix_left`, `infix_right`, or `env_body`, plus the index within that slot. | | `depth`, `mode` | Nesting depth and the content mode (`math` or `text`) the node lives in. | | `name`, `value` | The command or environment name, or the character or text value. | | `known` | Whether the knowledge base knows the command or environment, or `None` for other kinds. | | `group_kind`, `left`, `right` | For groups, whether they are explicit, implicit, delimited, or inline math, and the `\left` and `\right` delimiters. | The argument table has one row per slot of every command, declarative, infix, and environment, ordered by owner row and slot index. Its `owner` column is a row of the node table, `content` is the node-table row of the argument's content (or `-1`), and `present` tells you whether an optional slot was filled. `form` is the slot form (`mandatory`, `optional`, `star`, `group`, `delimited`, or `paired`), `value_kind` says what the slot carries, and `value`, `open`, and `close` hold literal values and paired boundaries. Field names stay snake_case in both languages. An incomplete document with `Error` nodes is still exportable. ## Count commands and argument forms [#count-commands-and-argument-forms] The loop below parses a few formulas and counts command names, unknown commands, and how often each command receives its optional argument. Rows in the argument table join back to the node table through `owner`, which is how an argument row finds its command name. **Python** ```python from collections import Counter import texform formulas = [ r"\frac{a}{b} + \sqrt[3]{x^2}", r"\sqrt{y} + \foo{x}", r"\mathbb{R}^n \to \mathbb{R}", r"\frac{1}{2} \sqrt[n]{z}", ] parser = texform.Parser() commands = Counter() # command name -> occurrences unknown = Counter() # names the knowledge base does not know optional_used = Counter() # command name -> filled optional slots max_depth = 0 for src in formulas: doc = parser.parse(src)["document"] if doc is None or doc.has_errors(): continue tree = doc.to_columnar() nodes, args = tree["nodes"], tree["args"] for kind, name, known in zip(nodes["kind"], nodes["name"], nodes["known"]): if kind == "Command": commands[name] += 1 if not known: unknown[name] += 1 # Argument rows point back to their owner row in the node table. for owner, form, present in zip(args["owner"], args["form"], args["present"]): if form == "optional" and present: optional_used[nodes["name"][owner]] += 1 max_depth = max(max_depth, max(nodes["depth"])) print("commands :", dict(commands)) print("unknown :", dict(unknown)) print("optional used:", dict(optional_used)) print("max depth :", max_depth) ``` **JavaScript** ```js import { Parser } from 'texform'; const formulas = [ '\\frac{a}{b} + \\sqrt[3]{x^2}', '\\sqrt{y} + \\foo{x}', '\\mathbb{R}^n \\to \\mathbb{R}', '\\frac{1}{2} \\sqrt[n]{z}', ]; const parser = new Parser(); const commands = new Map(); const unknown = new Map(); const optionalUsed = new Map(); const bump = (map, key) => map.set(key, (map.get(key) ?? 0) + 1); let maxDepth = 0; for (const src of formulas) { const doc = parser.parse(src).document; if (doc === null || doc.hasErrors()) continue; const { nodes, args } = doc.toColumnar(); nodes.kind.forEach((kind, row) => { if (kind === 'Command') { bump(commands, nodes.name[row]); if (!nodes.known[row]) bump(unknown, nodes.name[row]); } }); // Argument rows point back to their owner row in the node table. args.owner.forEach((owner, i) => { if (args.form[i] === 'optional' && args.present[i]) bump(optionalUsed, nodes.name[owner]); }); maxDepth = Math.max(maxDepth, ...nodes.depth); } console.log('commands :', JSON.stringify(Object.fromEntries(commands))); console.log('unknown :', JSON.stringify(Object.fromEntries(unknown))); console.log('optional used:', JSON.stringify(Object.fromEntries(optionalUsed))); console.log('max depth :', maxDepth); ``` ```text commands : {"frac":2,"sqrt":3,"foo":1,"mathbb":2,"to":1} unknown : {"foo":1} optional used: {"sqrt":2} max depth : 3 ``` In JavaScript the same program prints objects such as `{ frac: 2, sqrt: 3, foo: 1, mathbb: 2, to: 1 }`. Row indices restart at zero for every document, so keep per-document joins inside the loop, as above, and aggregate only the results across documents. ## Optional: Arrow tables [#optional-arrow-tables] In Python each table passes directly to `pyarrow.table`. TeXForm does not depend on pyarrow, so this needs `pip install pyarrow` and is only a convenience for people who already use Arrow or DataFrames. ```python import pyarrow as pa import texform doc = texform.Parser().parse(r"\frac{a}{b} + \sqrt[3]{x^2}")["document"] tree = doc.to_columnar() nodes = pa.table(tree["nodes"]) print(nodes.group_by("kind").aggregate([("depth", "max")]).sort_by("kind").to_pylist()) print(pa.table(tree["args"]).select(["owner", "index", "form", "present"]).to_pylist()[:2]) # Spans are keyed by path, not by table row. node = doc.find_commands("sqrt")[0] spans = {entry["id"]: entry["span"] for entry in doc.node_spans()} print(node.path(), spans[node.path()]) ``` ```text [{'kind': 'Char', 'depth_max': 3}, {'kind': 'Command', 'depth_max': 1}, {'kind': 'Root', 'depth_max': 0}, {'kind': 'Scripted', 'depth_max': 2}] [{'owner': 1, 'index': 0, 'form': 'mandatory', 'present': True}, {'owner': 1, 'index': 1, 'form': 'mandatory', 'present': True}] root.child.2 {'start': 14, 'end': 27} ``` ## Choose the right export [#choose-the-right-export] Use `to_columnar()` when you aggregate across many formulas: frequencies, depth statistics, argument-form counts, or feature extraction into a DataFrame. Use `to_syntax()` when you need a nested tree you can serialize to JSON, send over a process boundary, or rebuild with `Document.from_syntax`. Use `to_latex()` when you need text. The tables carry no source positions. Spans come from `node_spans()`, which returns byte spans keyed by the same rooted path that `node.path()` reports, as the last lines above show. Spans exist only for nodes parsed from source, and they are UTF-8 byte offsets as described in [Handle parse errors](https://texform.dev/docs/handle-parse-errors). ## Next steps [#next-steps] * [Document API](https://texform.dev/api/python/document) — to_columnar, to_syntax, node_spans, and the ColumnarTree, NodeTable, and ArgumentTable types. * [Node API](https://texform.dev/api/python/node) — Paths, slots, and argument reads for a single node. * [Normalize a corpus](https://texform.dev/docs/normalize-a-corpus) — Normalize formulas in bulk before you analyze them. * [Edit a document](https://texform.dev/docs/edit-a-document) — Change the tree once you know which nodes to target. --- # Configure output style Choose spaced or compact LaTeX output in the libraries, after normalization, and on the command line. https://texform.dev/docs/configure-output-style The serializer writes one of two output styles. `spaced`, the default, puts a space at every math token boundary and is byte-stable across releases, which suits datasets, caches, and tokenizers. `compact` writes LaTeX the way people usually type it, which suits display, review, and LLM prompts. The `style` option selects one for `to_latex()`, `to_tokenized_latex()`, and the module-level `serialize` function; it changes only the text layout, never the tree or the transform profile. ## Choose a style [#choose-a-style] **Python** ```python import texform doc = texform.Parser().parse(r"\begin{pmatrix}x_a^b&y\end{pmatrix}")["document"] print(doc.to_latex()) print(doc.to_latex(style="compact")) ``` **JavaScript** ```js import { Parser } from 'texform'; const doc = new Parser().parse(String.raw`\begin{pmatrix}x_a^b&y\end{pmatrix}`).document; console.log(doc.toLatex()); console.log(doc.toLatex({ style: 'compact' })); ``` ```text \begin {pmatrix} x _ { a } ^ { b } & y \end {pmatrix} \begin{pmatrix}x_a^b&y\end{pmatrix} ``` Compact drops every optional space, and the braces around a script that is a single TeX token, such as `x_a` or `x_\alpha`. It keeps the braces the syntax needs, as in `x^{10}`, and the spaces that separate tokens, as in `\alpha x`. After an unbraced script it adds one space before a letter or digit, so `x_i y` does not read as a subscript `iy`. Compact output is idempotent and parses back to the same tree, but its exact layout may still improve across releases. Store spaced output when you compare results across versions. Python takes `style` as a keyword argument; JavaScript takes it in an options object. Omitting it, or passing `None` or `null`, selects `spaced`, and an unknown value raises a configuration error instead of falling back to the default. ## Normalize to a style [#normalize-to-a-style] `normalize()` always returns spaced output: its configuration covers parsing and transformation, not serialization. To normalize into compact LaTeX, run the stages yourself and serialize the transformed document: **Python** ```python import texform engine = texform.TransformEngine(profile="corpus") doc = engine.parse(r"a \over b")["document"] engine.transform(doc) print(doc.to_latex(style="compact")) # \frac{a}{b} ``` **JavaScript** ```js import { TransformEngine } from 'texform'; const engine = new TransformEngine({ profile: 'corpus' }); const doc = engine.parse(String.raw`a \over b`).document; engine.transform(doc); console.log(doc.toLatex({ style: 'compact' })); // \frac{a}{b} ``` ## On the command line [#on-the-command-line] `texform normalize` takes the style as its own flag: ```console $ texform normalize --profile corpus --style compact 'a \over b' \frac{a}{b} ``` `--style` accepts `spaced` (the default) or `compact` and applies to plain and `--json` output alike. It is not a `--config` key, because `--config` holds the same normalize configuration as the libraries. `texform tokenize` always tokenizes the spaced form. A `texform serve` client sets `style` in the `config` of a [`configure`](https://texform.dev/docs/command-line#configure) request, beside `profile`, and the response's `resolved` reports the style in effect. ## Next steps [#next-steps] - [Serialization](https://texform.dev/docs/serialization) — Output channels, the spaced default, and the text-idempotency guarantee. - [Serialize API](https://texform.dev/api/python/serialize) — The serialize function and its style option. - [Edit a document](https://texform.dev/docs/edit-a-document) — Change the tree itself before you choose how to write it back out. - [Use with a renderer](https://texform.dev/docs/use-with-a-renderer) — Normalize formulas so MathJax or KaTeX can render them. --- # Inject custom commands Write an argspec, validate it, and register a custom command so the parser reads its arguments correctly. https://texform.dev/docs/inject-custom-commands The parser knows the argument structure of every command in its packages, but real corpora are full of project macros the knowledge base has never seen. Left unknown, a macro like `\mycmd[opt]{body}` parses as a bare command and its bracketed argument leaks out as loose tokens. You fix that by registering the command yourself: describe its signature as an argspec string, add it to a knowledge base as a context item, and a parser using that knowledge base reads its arguments exactly as if the command were built in. The workflow has three steps: write the argspec, check it with `validate_argspec`, then build a context item and pass it to the knowledge base the parser uses. The example registers `\mycmd` with the signature `o m` — one optional bracketed argument followed by one mandatory group. **Python** ```python import texform # 1. Write the signature and confirm it parses to the slots you expect. check = texform.validate_argspec("o m") print("valid:", check["valid"], "arg_count:", check["arg_count"]) # 2. Build a context item for the command. Python keys are snake_case. item = { "target": "command", "name": "mycmd", "kind": "prefix", "allowed_mode": "math", "argspec": "o m", } # 3. Add it to a knowledge base, build a parser on it, and parse a formula. kb = texform.KnowledgeBase(items=[item]) parser = texform.Parser(knowledge_base=kb) doc = parser.parse(r"\mycmd[opt]{body} + \mycmd{bare}")["document"] print("latex:", doc.to_latex()) for node in doc.find_commands("mycmd"): slots = [None if s is None else s["kind"] for s in node.arg_slots()] print("slots:", slots) ``` ```text valid: True arg_count: 2 latex: \mycmd [ o p t ] { b o d y } + \mycmd { b a r e } slots: ['Math', 'Math'] slots: [None, 'Math'] ``` **JavaScript** ```js import { KnowledgeBase, Parser, validateArgspec } from 'texform'; // 1. Write the signature and confirm it parses to the slots you expect. const check = validateArgspec('o m'); console.log('valid:', check.valid, 'argCount:', check.argCount); // 2. Build a context item for the command. JavaScript keys are camelCase. const item = { target: 'command', name: 'mycmd', kind: 'prefix', allowedMode: 'math', argspec: 'o m', }; // 3. Add it to a knowledge base, build a parser on it, and parse a formula. const knowledgeBase = new KnowledgeBase({ items: [item] }); const parser = new Parser({ knowledgeBase }); const doc = parser.parse('\\mycmd[opt]{body} + \\mycmd{bare}').document; console.log('latex:', doc.toLatex()); for (const node of doc.findCommands('mycmd')) { const slots = node.argSlots().map((s) => (s === null ? null : s.kind)); console.log('slots:', JSON.stringify(slots)); } ``` ```text valid: true argCount: 2 latex: \mycmd [ o p t ] { b o d y } + \mycmd { b a r e } slots: ["Math","Math"] slots: [null,"Math"] ``` The signature is honored: `\mycmd[opt]{body}` fills both slots, while `\mycmd{bare}` leaves the optional slot empty (`None` in Python, `null` in JavaScript) and fills only the mandatory one. The parser bound `[opt]` as the optional argument instead of letting it fall through as stray characters, which is the whole point of registering the command. Commands are not the only thing you can inject. A context item is a tagged record, and the `target` field selects one of three kinds: `"command"` as above, `"environment"` for a custom `\begin{...}` block (which also takes a `body_mode`, since an environment body can be math or text), and `"delimiter"` for a delimiter-control name usable after `\left` and `\right`. They all go into the same `items` list of the `KnowledgeBase` constructor. ```python import texform item = { "target": "environment", "name": "myenv", "allowed_mode": "math", "body_mode": "math", "argspec": "m", } kb = texform.KnowledgeBase(items=[item]) doc = texform.Parser(knowledge_base=kb).parse(r"\begin{myenv}{c} x \end{myenv}")["document"] print(doc.to_latex()) ``` ```text \begin {myenv} { c } x \end {myenv} ``` What you cannot inject is a rewrite rule. Context items extend the parser's knowledge of *what a command's arguments look like*; they do not tell the transform engine what to turn that command into. The rule set is compiled in, and the only user-facing control over it is switching rules off — see [Disable a rewrite rule](https://texform.dev/docs/disable-a-rule). For the full argspec notation — every slot kind, delimited forms, and the extensions to xparse — see the [argspec reference](https://texform.dev/docs/argspec); for the context-item shape and construction options, see the [KnowledgeBase API](https://texform.dev/api/python/knowledge-base). ## Next steps [#next-steps] - [Argspec reference](https://texform.dev/docs/argspec) — Every slot kind, delimited form, and TeXForm extension to xparse notation. - [KnowledgeBase API](https://texform.dev/api/python/knowledge-base) — Context-item shapes and the rest of the knowledge-base construction options. - [Disable a rewrite rule](https://texform.dev/docs/disable-a-rule) — The one piece of the transform engine you can adjust from the outside. - [Clean OCR/MER output](https://texform.dev/docs/clean-ocr-mer-output) — Find the unknown commands worth registering in the first place. --- # Compare formulas Use equiv-profile normalization for cheap spelling comparison, and a CAS for true mathematical equivalence. https://texform.dev/docs/compare-formulas > **TeXForm handles spelling, not mathematics** > > TeXForm's role in formula comparison is the normalization layer, not the equivalence engine. It canonicalizes how a formula is written and stops there. Deciding whether two formulas are mathematically equal is a job for a computer algebra system, and layer 2 below shows how to hand off to one. Comparing two LaTeX formulas usually means one of two different questions, and it helps to separate them. The cheap question is whether two formulas are the same up to spelling — `\cfrac{a}{b}` and `\frac{a}{b}` denote the same fraction written two ways. The expensive question is whether two formulas are mathematically equal — `x + x` and `2x` are equal but structurally different. TeXForm answers the first directly and is deliberately not in the business of answering the second. **Layer 1 — spelling equality via the `equiv` profile.** Normalize both formulas under the `equiv` profile and compare the resulting strings. This profile canonicalizes equivalent spellings hard enough that surface differences collapse, so a string comparison resolves them with no extra dependencies. **Python** ```python import texform equiv = texform.TransformEngine(profile="equiv") corpus = texform.TransformEngine(profile="corpus") def same_form(engine, a, b): return engine.normalize(a) == engine.normalize(b) for a, b in [(r"\cfrac{a}{b}", r"\frac{a}{b}"), (r"\widehat{x}", r"\hat{x}"), (r"x + x", r"2x")]: print(f"{a!r:16} vs {b!r:14} equiv={same_form(equiv, a, b)} corpus={same_form(corpus, a, b)}") ``` ```text '\\cfrac{a}{b}' vs '\\frac{a}{b}' equiv=True corpus=False '\\widehat{x}' vs '\\hat{x}' equiv=True corpus=False 'x + x' vs '2x' equiv=False corpus=False ``` **JavaScript** ```js import { TransformEngine } from 'texform'; const equiv = new TransformEngine({ profile: 'equiv' }); const corpus = new TransformEngine({ profile: 'corpus' }); const sameForm = (engine, a, b) => engine.normalize(a) === engine.normalize(b); for (const [a, b] of [['\\cfrac{a}{b}', '\\frac{a}{b}'], ['\\widehat{x}', '\\hat{x}'], ['x + x', '2x']]) { console.log(a, 'vs', b, 'equiv =', sameForm(equiv, a, b), 'corpus =', sameForm(corpus, a, b)); } ``` The `corpus` column is why `equiv` exists. `\cfrac` and `\widehat` are notation choices a training label should keep — `corpus` preserves the continued-fraction styling and the wide hat, so the two spellings stay distinct there. `equiv` deliberately discards that distinction (`ams/cfrac-to-frac` and `base/widehat-to-hat` are the rules doing it) because a comparison key should not care how wide the hat was. The third row states the other boundary just as plainly: `x + x` and `2x` do not compare equal, because `equiv` normalizes form, not meaning. It never performs algebra — it will not factor, expand, or evaluate. Treat layer 1 as a fast, zero-dependency filter that resolves spelling differences and nothing more, and reach for `equiv` output as a comparison key or fingerprint rather than as a label you would show a reader. **Layer 2 — mathematical equivalence via a CAS.** For the cases layer 1 reports as different, hand the normalized strings to a computer algebra system. The snippet below sketches the hookup with SymPy; a math-recognition evaluation harness such as Math-Verify wraps the same idea with parsing and timeout handling built in. ```python # Illustrative — requires `pip install sympy antlr4-python3-runtime`. import texform from sympy.parsing.latex import parse_latex engine = texform.TransformEngine(profile="equiv") def math_equal(a, b): expr_a = parse_latex(engine.normalize(a)) expr_b = parse_latex(engine.normalize(b)) return (expr_a - expr_b).simplify() == 0 # math_equal(r"x + x", r"2x") -> True ``` Normalizing with `equiv` first does real work for the CAS: it removes the spelling variation a LaTeX parser would otherwise choke on, so the CAS only has to reason about genuine mathematical difference. For how the `equiv` profile canonicalizes and why it stops short of mathematical reasoning, see [Transforms](https://texform.dev/docs/transforms). For how this compares with string normalizers used in answer grading, see [How TeXForm compares](https://texform.dev/docs/comparison). ## Next steps [#next-steps] - [Transforms](https://texform.dev/docs/transforms) — What each profile canonicalizes, and the rule levels behind the equiv target. - [Rule reference](https://texform.dev/rules) — Every rule the equiv profile adds, with its level and fidelity. - [Normalize a corpus](https://texform.dev/docs/normalize-a-corpus) — Apply one engine across a dataset when you are deduplicating at scale. --- # Disable a rewrite rule Turn off a single rewrite rule by key when a profile normalizes more than you want, and see what disabling does and does not change. https://texform.dev/docs/disable-a-rule Sometimes a profile is right about almost everything and wrong about one rule. You want `corpus` normalization across the board, but your downstream consumer needs `\dv` to survive as written, or a particular rewrite conflicts with a convention in your dataset. Rather than dropping to a weaker profile and losing everything else, you disable that one rule by key when you construct the engine. Rules are identified by a `package/rule-name` key, the same key that appears in the transform report and in the [rule reference](https://texform.dev/rules). Pass the keys you want off to the engine constructor; the engine compiles its plan without them and behaves normally otherwise. The examples load `physics` explicitly, since it is not in the default package set. **Python** ```python import texform kb = texform.KnowledgeBase(packages=["base", "ams", "physics"]) default = texform.TransformEngine(profile="corpus", knowledge_base=kb) without_dv = texform.TransformEngine( profile="corpus", knowledge_base=kb, disable_rules=["physics/dv-to-frac-d"], ) for src in [r"\dv{f}{x}", r"\pdv{f}{x}", r"a \over b"]: print(f"{src!r:16} default={default.normalize(src)!r}") print(f"{'':16} disabled={without_dv.normalize(src)!r}") ``` ```text '\\dv{f}{x}' default='\\frac { \\mathrm { d } f } { \\mathrm { d } x }' disabled='\\dv { f } { x }' '\\pdv{f}{x}' default='\\frac { \\partial f } { \\partial x }' disabled='\\frac { \\partial f } { \\partial x }' 'a \\over b' default='\\frac { a } { b }' disabled='\\frac { a } { b }' ``` **JavaScript** ```js import { KnowledgeBase, TransformEngine } from 'texform'; const knowledgeBase = new KnowledgeBase({ packages: ['base', 'ams', 'physics'] }); const defaultEngine = new TransformEngine({ profile: 'corpus', knowledgeBase }); const withoutDv = new TransformEngine({ profile: 'corpus', knowledgeBase, disableRules: ['physics/dv-to-frac-d'], }); for (const src of ['\\dv{f}{x}', '\\pdv{f}{x}', 'a \\over b']) { console.log(src, 'default =', defaultEngine.normalize(src)); console.log(' '.repeat(src.length), 'disabled =', withoutDv.normalize(src)); } ``` The disabled rule is the only thing that changed. `\dv{f}{x}` now passes through as the author wrote it, while `\pdv` still expands — it is a different rule (`physics/pdv-to-frac-partial`) and was never disabled — and `a \over b` still modernizes, because that belongs to `base`. Disabling is surgical, not a downgrade of the whole profile. The transform report confirms it, which is the reliable way to check that a key you typed actually matched a rule: ```python import texform kb = texform.KnowledgeBase(packages=["base", "ams", "physics"]) default = texform.TransformEngine(profile="corpus", knowledge_base=kb) without_dv = texform.TransformEngine( profile="corpus", knowledge_base=kb, disable_rules=["physics/dv-to-frac-d"] ) def fired(engine, src): report = engine.normalize_with_report(src)["report"] return [r["key"] for r in report["rewrite"]["rules"] if r["applied_count"] > 0] print("default :", fired(default, r"\dv{f}{x}")) print("disabled:", fired(without_dv, r"\dv{f}{x}")) ``` ```text default : ['physics/dv-to-frac-d'] disabled: [] ``` ## What disabling a rule does not do [#what-disabling-a-rule-does-not-do] Three boundaries are worth stating plainly, because each is a thing people reasonably expect and TeXForm does not offer. **You cannot register your own rewrite rule at runtime.** The rule set is compiled into the engine, and the public API exposes no way to add to it. `disable_rules` subtracts; nothing adds. Contributing a rule means adding it to the transform crate — see [Rules and the transform contract](https://texform.dev/docs/rule-contract). Injecting *parser* knowledge is a separate capability that does exist, covered in [Inject custom commands](https://texform.dev/docs/inject-custom-commands); it teaches the parser a command's argument structure, which is not the same as teaching the engine a rewrite. **Disabling a phase is not the same as disabling a rule.** The per-run config can switch off a whole pipeline phase, which is a much blunter instrument — turning off Rewrite drops every rule at once: **Python** ```python import texform kb = texform.KnowledgeBase(packages=["base", "ams", "physics"]) engine = texform.TransformEngine(profile="corpus", knowledge_base=kb) print(engine.normalize(r"\dv{f}{x}", rewrite={"enabled": False})) ``` ```text \dv { f } { x } ``` **JavaScript** ```js import { KnowledgeBase, TransformEngine } from 'texform'; const knowledgeBase = new KnowledgeBase({ packages: ['base', 'ams', 'physics'] }); const engine = new TransformEngine({ profile: 'corpus', knowledgeBase }); console.log(engine.normalize('\\dv{f}{x}', { rewrite: { enabled: false } })); ``` That output looks identical to disabling the single rule, which is exactly the trap: on this one input the two are indistinguishable, but across a corpus the phase switch also suppresses `\over`, `\pdv`, and everything else. Use `disable_rules` when you mean one rule. **Disabling a rule does not change the profile's rule level.** A profile selects which levels of rules participate; `disable_rules` removes individual keys from whatever that selection produced. If you find yourself disabling many rules to approximate a different target, a different profile is probably the right answer — see the profile comparison in [Transforms](https://texform.dev/docs/transforms). ## Next steps [#next-steps] - [Rule reference](https://texform.dev/rules) — Browse every rewrite rule with its key, level, and the forms it eliminates. - [Transforms](https://texform.dev/docs/transforms) — Profiles, the phase pipeline, and how to read the transform report. - [Inject custom commands](https://texform.dev/docs/inject-custom-commands) — Teach the parser a macro's argument structure with a context item. - [Rules and the transform contract](https://texform.dev/docs/rule-contract) — How rules declare their level, fidelity, and eliminated forms. --- # Use TeXForm with MathJax or KaTeX Normalize LaTeX so a plain MathJax or KaTeX deployment can render it, expanding physics macros that vanilla renderers do not know. https://texform.dev/docs/use-with-a-renderer If you display user-supplied or scraped LaTeX in a browser, you have probably seen formulas that fail to render because the author used a macro their package provides and your renderer does not. `\dv{f}{x}` is valid LaTeX in a document that loads `physics`, and it is nothing at all to a stock MathJax or KaTeX deployment. TeXForm fixes that before display: normalize under a profile that expands convenience macros, and hand the resulting string to the renderer you already use. TeXForm is not a MathJax or KaTeX plugin, and it does not render anything. It is a preprocessing step. Input is a LaTeX string, output is a LaTeX string, and the renderer stays exactly where it is in your stack. ## Expand macros the renderer does not know [#expand-macros-the-renderer-does-not-know] The `faithful` and `corpus` profiles both expand convenience macros into the universal forms a vanilla renderer understands. `faithful` is the conservative choice for display: it keeps the rendered formula the same and only replaces markup that needs a package. `physics` is not in the default package set, so an engine that should expand its macros names it. **Python** ```python import texform kb = texform.KnowledgeBase(packages=["base", "ams", "physics"]) engine = texform.TransformEngine(profile="faithful", knowledge_base=kb) for src in [r"\dv{f}{x}", r"\pdv{f}{x}", r"\abs{x}", r"\norm{v}", r"a \over b", r"{\bf x}"]: print(f"{src!r:14} -> {engine.normalize(src)!r}") ``` ```text '\\dv{f}{x}' -> '\\frac { \\mathrm { d } f } { \\mathrm { d } x }' '\\pdv{f}{x}' -> '\\frac { \\partial f } { \\partial x }' '\\abs{x}' -> '\\left | x \\right |' '\\norm{v}' -> '\\left \\| v \\right \\|' 'a \\over b' -> '\\frac { a } { b }' '{\\bf x}' -> '\\mathbf { x }' ``` **JavaScript** ```js import { KnowledgeBase, TransformEngine } from 'texform'; const knowledgeBase = new KnowledgeBase({ packages: ['base', 'ams', 'physics'] }); const engine = new TransformEngine({ profile: 'faithful', knowledgeBase }); for (const src of ['\\dv{f}{x}', '\\pdv{f}{x}', '\\abs{x}', '\\norm{v}', 'a \\over b', '{\\bf x}']) { console.log(src, '->', engine.normalize(src)); } ``` Every output on the right is plain `base`/`ams` LaTeX. `\frac`, `\partial`, `\left`, `\right`, and `\mathbf` are all things a default MathJax or KaTeX build renders without extra configuration, while none of the inputs on the left would have rendered at all. Legacy syntax like `\over` is modernized by every profile, so that one comes along for free. ## Wire it into the render path [#wire-it-into-the-render-path] The normalization is one call in front of whatever you already do. Reuse a single engine — construction compiles the rule plan, so building one per formula is wasted work — and fall back to the original string when normalization declines, so a malformed formula degrades to the renderer's own error handling rather than disappearing. ```js import { KnowledgeBase, TexformParseError, TransformEngine } from 'texform'; import katex from 'katex'; const knowledgeBase = new KnowledgeBase({ packages: ['base', 'ams', 'physics'] }); const engine = new TransformEngine({ profile: 'faithful', knowledgeBase }); function render(element, src) { let source = src; try { source = engine.normalize(src); } catch (err) { if (!(err instanceof TexformParseError)) throw err; // Incomplete formula: let the renderer report it against the original text. } katex.render(source, element, { throwOnError: false }); } ``` ## Choosing a profile for display [#choosing-a-profile-for-display] `faithful` is the default recommendation for rendering because it is defined to preserve the rendered result. `corpus` expands further and may collapse layout variants that a reader would notice, which is the right trade for training data and the wrong one for a page someone is reading. `equiv` goes further still and discards visually salient notation — it rewrites `\cfrac` to `\frac`, for instance — so it belongs in comparison and deduplication pipelines, not in a render path. The profile table in [Transforms](https://texform.dev/docs/transforms) lays out the targets side by side. ## How the knowledge base relates to renderers [#how-the-knowledge-base-relates-to-renderers] MathJax, KaTeX, and XeTeX appear in TeXForm's development as *validators*, not dependencies. Every argument specification in the knowledge base is checked by rendering it against all three engines, which is how the project knows a signature is right rather than merely plausible. That is also why the coverage data on this site reports support per engine: the same formula can be renderable in one and not another. Two pages cover that ground. [Renderer validation](https://texform.dev/docs/renderer-validation) explains the methodology and how to read the full, partial, and none verdicts. The [renderer coverage reference](https://texform.dev/latex/renderers) lets you look up a specific command and see which engines accept it — useful when a formula still fails after normalization and you need to know whether the command is supported by your renderer at all. ## Next steps [#next-steps] - [Renderer coverage](https://texform.dev/latex/renderers) — Look up which of MathJax, KaTeX, and XeTeX support a given command. - [Renderer validation](https://texform.dev/docs/renderer-validation) — How specifications are validated by rendering against three engines. - [Transforms](https://texform.dev/docs/transforms) — What each profile targets and how the phase pipeline produces it. - [Normalize a corpus](https://texform.dev/docs/normalize-a-corpus) — Apply the same engine across a dataset with error handling and statistics. --- # Troubleshooting Fix common TeXForm failures: WASM loading, ConfigError on config shapes, mismatched knowledge bases, incomplete trees, and package conflicts. https://texform.dev/docs/troubleshooting When TeXForm refuses to do something, it almost always says so through a structured exception rather than a silent wrong answer, so the error class and its message are the fastest route to the cause. This page lists the failures people actually hit, each as a symptom you can recognize and the check that resolves it. The Python messages below are real interpreter output; the JavaScript counterparts throw the `Texform*` class named in each section. ## If importing `texform` fails in a bundler, check top-level await and `.wasm` assets [#if-importing-texform-fails-in-a-bundler-check-top-level-await-and-wasm-assets] The npm package is WebAssembly, and it ships two entry points for loading it. The default `texform` import resolves to the Node entry under Node.js and to the bundler entry in browser-oriented bundlers; `texform/node` and `texform/bundler` force one explicitly. The bundler entry initializes the WebAssembly module at module load time, so it needs a bundler that supports top-level `await` and can serve `.wasm` as an asset — Vite or webpack 5, for example. A build that fails on top-level `await`, or a runtime 404 for the `.wasm` file, means the bundler is not configured for one of those two things. A test runner or server-side script that picked the bundler entry by mistake is the other common shape of this failure, and the fix is to import the Node entry directly. ```js // In Node.js — skips the bundler entry and its top-level await entirely. import { TransformEngine } from 'texform/node'; // In a browser bundle — requires top-level await and .wasm asset support. import { TransformEngine } from 'texform/bundler'; ``` ## If you get `ConfigError` or `TexformConfigError`, check the shape of the config you passed [#if-you-get-configerror-or-texformconfigerror-check-the-shape-of-the-config-you-passed] Configuration is validated strictly, and the message carries the field path that failed. Three shapes account for nearly every case. **An unknown or misspelled key.** Per-call options are an overlay, and the validator lists the keys it would have accepted: **Python** ```python import texform engine = texform.TransformEngine(profile="corpus") try: engine.normalize(r"a \over b", rewrite={"enable": False}) except texform.ConfigError as exc: print("ConfigError:", exc) # The accepted spelling: print(engine.normalize(r"a \over b", rewrite={"enabled": False})) ``` ```text ConfigError: invalid normalize overrides: rewrite.enable: unknown field `enable`, expected `enabled` or `max_iterations` a \over b ``` **JavaScript** ```js import { TransformEngine, TexformConfigError } from 'texform'; const engine = new TransformEngine({ profile: 'corpus' }); try { engine.normalize('a \\over b', { rewrite: { enable: false } }); } catch (exc) { if (exc instanceof TexformConfigError) console.log(exc.message); } // The accepted spelling: console.log(engine.normalize('a \\over b', { rewrite: { enabled: false } })); ``` **A snake_case key in JavaScript.** The JavaScript overlay is camelCase throughout; `reject_unknown` is rejected where `rejectUnknown` is accepted. Enum *values* stay snake_case (`'sub_first'`), which is the one place the two conventions meet in a single object. Python is the mirror image: keys are snake_case there. **A dict used where a complete config object belongs.** In Python the second positional argument is a complete `TransformConfig` (or `ParseConfig` on `parse`), not a mapping. Pass adjustments as keyword overlays instead, and note that nested overlay values must be plain dicts rather than phase-config instances: ```python import texform engine = texform.TransformEngine(profile="corpus") try: engine.normalize(r"a \over b", {"rewrite": {"enabled": False}}) except texform.ConfigError as exc: print("ConfigError:", exc) try: engine.normalize(r"a \over b", rewrite=texform.RewriteConfig(enabled=False, max_iterations=32)) except texform.ConfigError as exc: print("ConfigError:", exc) ``` ```text ConfigError: config must be a TransformConfig; pass overrides as keyword arguments (use **overrides for a dict) ConfigError: invalid normalize overrides: rewrite: unsupported value of type `RewriteConfig`; pass a dict ``` If you do want a complete object, build one with `TransformConfig.corpus()` (or the matching profile classmethod) and adjust its fields, rather than constructing `TransformConfig` from scratch — every phase is required there. ## If `transform` rejects a document, check which knowledge base it was built on [#if-transform-rejects-a-document-check-which-knowledge-base-it-was-built-on] In-place transformation requires a document built on the engine's own knowledge-base instance. Knowledge bases compare by identity, so a document from a parser with a separately constructed `KnowledgeBase` is foreign to the engine — even when both load the same packages — and is rejected instead of being silently normalized against different knowledge. ```python import texform engine = texform.TransformEngine(profile="corpus") other = texform.Parser(knowledge_base=texform.KnowledgeBase()) foreign = other.parse(r"a \over b")["document"] try: engine.transform(foreign) except texform.TransformError as exc: print("TransformError:", exc) # Parse through the engine (or share one KnowledgeBase), and the same flow works. own = engine.parse(r"a \over b")["document"] engine.transform(own) print(own.to_latex()) ``` ```text TransformError: document and transform engine use different knowledge-base instances; build one KnowledgeBase and share it with the parser, engine, and documents \frac { a } { b } ``` The one-shot `normalize` never hits this, because it parses and transforms in a single call. Reach for `engine.parse` whenever you need the tree in between. ## If normalization rejects a formula, check whether the tree is complete [#if-normalization-rejects-a-formula-check-whether-the-tree-is-complete] Normalization is gated on a complete tree: a formula with holes in it has no meaningful canonical form, so the engine declines rather than guessing. The two entry points report this differently, and it is worth handling both. ```python import texform engine = texform.TransformEngine(profile="corpus") try: engine.normalize(r"x^") except texform.ParseError as exc: print("ParseError:", exc) print(" diagnostics:", exc.diagnostics) print(" partial document recovered:", exc.document is not None) partial = engine.parse(r"x^")["document"] print("has_errors:", partial.has_errors()) try: engine.transform(partial) except texform.TransformError as exc: print(f"{type(exc).__name__}:", exc) ``` ```text ParseError: parse produced an incomplete document diagnostics: [{'kind': 'raw-expected-found', 'message': 'Missing superscript content', 'span': {'start': 2, 'end': 2}, 'expected': ['superscript content'], 'found': None, 'contexts': []}] partial document recovered: True has_errors: True TransformError: cannot transform a document with parse errors ``` The two classes answer different questions, which is why they stay distinct. `normalize` could not get a complete tree out of the source, so it raises `ParseError` and carries the diagnostics and the partial document — a corpus loop can bucket the input for repair instead of discarding it. `transform` was handed a tree that does not meet its precondition, so it raises `TransformError`, the same class you get for a foreign document. In JavaScript the matching classes are `TexformParseError` (with the same `diagnostics` and `document` fields) and `TexformTransformError`. For splitting these buckets deliberately, see [Clean OCR/MER output](https://texform.dev/docs/clean-ocr-mer-output). > Both languages name the base class `TexformError`, and catching it catches every library error. That is the safe choice in a batch loop where you care more that a formula failed than exactly why. ## If a package name is rejected, check it against `list_packages()` [#if-a-package-name-is-rejected-check-it-against-list_packages] Package and profile names are validated at construction, so a typo fails immediately rather than quietly producing a knowledge base with a hole in it. ```python import texform try: texform.KnowledgeBase(packages=["physik"]) except texform.ConfigError as exc: print("ConfigError:", exc) try: texform.TransformEngine(profile="canonical") except texform.ConfigError as exc: print("ConfigError:", exc) print([p["name"] for p in texform.list_packages()]) ``` ```text ConfigError: unknown package: physik ConfigError: unknown transform profile: canonical ['ams', 'base', 'bboldx', 'boldsymbol', 'braket', 'color', 'physics', 'textmacros'] ``` The profile names are `authoring`, `faithful`, `corpus`, and `equiv`. ## If `\ket` and `\bra` normalize the wrong way, check `braket` against `physics` [#if-ket-and-bra-normalize-the-wrong-way-check-braket-against-physics] `list_packages()` reports the whole catalog, but the default knowledge base loads the default runtime set, which consists of: `base`, `ams`, `braket`, `textmacros`, `bboldx`, `boldsymbol`, and `color`. `physics` is left out because it defines `\bra`, `\ket`, and friends with different semantics than `braket` does, and only one of the two can own those names. The consequence is quiet rather than loud: `\ket{\psi}` still parses cleanly by default, because `braket` claims it. You only notice when the normalized output is not the notation you expected. ```python import texform default = texform.TransformEngine(profile="corpus") kb = texform.KnowledgeBase(packages=["base", "ams", "physics"]) physics = texform.TransformEngine(profile="corpus", knowledge_base=kb) print("braket (default):", default.normalize(r"\ket{\psi}")) print("physics :", physics.normalize(r"\ket{\psi}")) ``` ```text braket (default): \ket { \psi } physics : \left \vert \psi \right \rangle ``` To get physics semantics, name the packages explicitly and include `physics`. Listing both loads without error; `physics` wins the contested names, while braket-only commands such as `\Ket` stay available. ## If an edit is rejected, check node ownership and whether the tree is read-only [#if-an-edit-is-rejected-check-node-ownership-and-whether-the-tree-is-read-only] Two editing guards produce most failures. A node belongs to the document that created it, and a tree that came back with parse errors is read-only for its whole lifetime. Both report `EditError` in Python and `TexformEditError` in JavaScript, and both reject the edit before any tree is modified. ```python import texform doc_a = texform.Parser().parse(r"x + y")["document"] doc_b = texform.Parser().parse(r"a + b")["document"] try: doc_a.append_child(doc_a.root(), doc_b.create_char("z")) except texform.EditError as exc: print(f"{type(exc).__name__}:", exc) read_only = texform.Parser().parse(r"x^")["document"] try: read_only.append_child(read_only.root(), read_only.create_char("y")) except texform.EditError as exc: print(f"{type(exc).__name__}:", exc) ``` ```text EditError: node belongs to a different document EditError: document is read-only ``` Build replacement content with the `create_*` constructors on the document you are editing, and it stays owned by the right tree. See [Edit a document](https://texform.dev/docs/edit-a-document) for the full workflow. ## Next steps [#next-steps] - [Handle parse errors](https://texform.dev/docs/handle-parse-errors) — Dispatch on the three parse states and read diagnostics and spans. - [Clean OCR/MER output](https://texform.dev/docs/clean-ocr-mer-output) — Bucket noisy recognition output into repairable and discardable input. - [Parsing](https://texform.dev/docs/parsing) — The knowledge base, the strictness knobs, and the parse-state model behind these errors. - [Transforms](https://texform.dev/docs/transforms) — Profiles, the phase pipeline, and what the transform report tells you. --- # Architecture Crate layout, the stability boundary, the processing pipeline, three tree representations, and how the bindings work. https://texform.dev/docs/architecture Read this if you contribute to TeXForm, integrate it and want to know exactly what the public API guarantees, or are simply curious how a formula moves through the system. This page covers the *why* of the structure: where the stability boundary sits, the pipeline a formula travels through, the three tree representations and why they are distinct, and how the language bindings expose one Rust core. This page is the site-facing account of the model that the repository's [`ARCHITECTURE.md`](https://github.com/texform-dev/texform/blob/main/ARCHITECTURE.md) defines for contributors reading the source. Where the two differ in wording, the repository document is the normative one; the [source map](#source-map) at the end of this page lists the files each section is drawn from. ## Crate layout and the stability boundary [#crate-layout-and-the-stability-boundary] TeXForm is a Rust workspace of small, single-purpose crates. Only one of them — `texform` — carries a public stability guarantee. Every other crate is an internal implementation detail whose API may change without notice. External code must integrate through the `texform` facade. Python and WebAssembly bindings use the public texform facade. Internal dependencies are transform, core, knowledge, knowledge-macros, argspec, and interface; regression is internal tooling. Arrows point from a crate to the crates it depends on. Everything below `texform` is internal; the facade is the only general-purpose Rust crate external code should depend on. The binding crates are published language surfaces, but they may still depend on internal helper crates for host-language glue such as generated TypeScript shapes and option conversion. | Crate | Responsibility | | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `texform` | Public facade. Re-exports the stable surface: `Parser`, `TransformEngine`, `Document`, `ParseResult`, serialization, `validate_argspec`, analysis helpers, and package metadata. The only general-purpose Rust crate other code should depend on. | | `texform-core` | The parser, the internal `Ast` arena, the serializer, and the public `Document` DOM layer. | | `texform-transform` | The phase-oriented rewrite and normalization engine that operates on a parsed tree. | | `texform-knowledge` | The knowledge base: commands, environments, characters, and delimiters — which names are known, in which package and mode, and what argument shapes they take. | | `texform-knowledge-macros` | Procedural macros used by the knowledge base, including compile-time argument-specification parsing for generated records. | | `texform-argspec` | An xparse-style argument-specification parser used to describe command signatures. | | `texform-interface` | Dependency-free shared types, most importantly `SyntaxNode`, the lossless parse snapshot. | | `texform-python`, `texform-wasm` | Language bindings that expose the shared facade model to Python and WebAssembly. | | `texform-regression` | Corpus regression and data-product tooling for parser regression, transform-contract checking, and counter-map generation. Internal tooling, not part of the public API. | The facade deliberately does **not** re-export the internal `Ast`, `Node`, or arena types. Users get a single editable tree type — `Document` — and never touch the panic-contract arena underneath it. In each language the package you install is `texform` (`pip install texform`, `npm install texform`, `cargo add texform`); the internal `texform-*` crates are not meant to be a direct dependency. ## The processing pipeline [#the-processing-pipeline] A formula flows through the system in one direction: parsing produces an editable document, and that document supports four separate operations — text output, tokenized output, transformation, and a syntax snapshot. LaTeX source → Parser and knowledge base → SyntaxNode snapshot → editable Document. The Document supports text output, typed tokens, normalization, and structured snapshots. The parser produces a `SyntaxNode` first. TeXForm then converts that snapshot into a `Document`, which is the tree users read, edit, serialize, and transform. The parser-driven path also carries source spans across into the `Document`. A `SyntaxNode` is never edited directly: it is the wire format, and `Document` is the working format. `to_latex()` and `to_tokenized_latex()` are the same serializer traversal: the tokenized form turns on a recorder that emits typed fragments with UTF-8 byte spans into the identical LaTeX string, so the two never disagree about the text. See [serialization](https://texform.dev/docs/serialization) for when each one is worth reaching for. ## Three tree representations [#three-tree-representations] The same formula is modeled by three trees with deliberately separated roles. Confusing them is the most common source of design mistakes, so the boundaries are explicit. * **`SyntaxNode`** (from `texform-interface`) is the lossless, immutable parse snapshot. It is the parser's stage-1 output and the single serde DTO — it derives `Serialize`/`Deserialize` and backs JSON snapshots, Python dictionaries, JavaScript objects, and test fixtures. It can represent a partial parse (it may contain `Error` nodes) but carries no editing behavior. * **`Document`** (from `texform-core`, re-exported by the facade) is the public, editable DOM-style tree. It is what users construct, query, mutate, serialize, and transform. Reads go through lightweight `NodeRef` handles; edits are fallible and return a `Result` with an `EditError` on misuse. * **`Ast`** is the internal arena tree that the transform engine and serializer operate on. It is a *panic-contract* type: its methods panic on misuse, because misuse means an internal invariant was violated, not that a user supplied bad input. It is **not** part of the public API. `Document` wraps it and exposes a fallible surface over it, so no arena panic can reach a caller on a user-input-driven path. `Document::from_syntax` and `Document::to_syntax` bridge the snapshot and the working tree in both directions, and the conversion is symmetric over every node kind, including `Error` and `Prime`. Because a snapshot can come from outside the parser, `from_syntax` validates external input and rejects invalid syntax with a structured error rather than letting the internal arena panic. ### Structural validity versus semantic completeness [#structural-validity-versus-semantic-completeness] Two notions of "valid" are kept strictly apart, because they answer different questions. * **Structural validity** is about the shape of the arena: parent links are consistent, slots hold the right node kinds, an environment body is a group, the root is unique and parentless, and there are no cycles. `Ast` and `Document` maintain structural validity **at all times**. Crucially, promoting parse errors to first-class `Error` *leaves* means a tree containing errors is still structurally valid. * **Semantic completeness** is about whether the tree contains any `Error` placeholder nodes. This is a separate, O(1)-queryable property exposed by `has_errors()` — not a structural invariant. The method is named for the concrete fact it reports, so read-only checks and transform gates read without a double negative. This mirrors MathJax: its rendered MathML embeds error nodes for unparseable fragments, the output tree stays well-formed, and the renderer simply handles them. TeXForm takes the same approach — `Error` is a first-class node kind, the tree is always structurally valid, and "does it contain errors" is a cheap signal downstream consumers can check without validating the whole tree. A tree that `has_errors()` is read-only: there is nothing useful to do with a tree full of holes except inspect it, so fixing read-only-ness at construction keeps the editing contract simple. `TransformEngine.transform` also requires a `Document` produced by that engine's parser; documents from another parser or rebuilt from syntax are rejected. For the three honest `ParseResult` states and how errors round-trip, see [the parsing concept page](https://texform.dev/docs/parsing). ## Language bindings [#language-bindings] The Python and WebAssembly bindings expose live `Document` and `Node` handles rather than copying trees across the language boundary. * A binding `Node` is a cheap handle — a shared reference to its owning document plus an internal node id. All reads and edits delegate back to the document; the tree is never cloned across the boundary. * The core `Document` stays a plain owned Rust value with no interior mutability. Sharing is provided only at the binding layer, using each runtime's native mechanism: a reference-counted cell on Python, and `Rc>` on WASM. Direct Rust users never pay for the bindings' sharing needs. * Borrow conflicts and misuse surface as structured host-language exceptions, never as a panic crossing the FFI boundary. A read-only (error) document raises a read-only exception, and an edit mixing nodes from two documents is rejected before it reaches the core. ### Casing conventions [#casing-conventions] The bindings present the same core through each language's idioms, and the casing rule is precise. * **Python** exposes the canonical Rust snake_case form directly. Methods and option keys are snake_case (`has_errors()`, `to_latex()`, `finalize_ast`), and structured data — parse results, reports, lookup metadata, argspec validation results — is returned as plain `dict` values (typed with `TypedDict` in the bundled `.pyi` stubs). * **JavaScript** provides the camelCase view of the same shapes: `hasErrors()`, `toLatex()`, `rejectUnknown`, `finalizeAst`. The public TypeScript declarations ship with the npm package. ```python # Python: snake_case methods, dict-shaped data import texform engine = texform.TransformEngine(profile="corpus") print(engine.normalize(r"a \over b")) # \frac { a } { b } reported = engine.normalize_with_report(r"a \over b") print(reported["report"]["rewrite"]["iterations"]) # 2 ``` ```ts // JavaScript: camelCase methods, the same shapes import { TransformEngine } from "texform"; const engine = new TransformEngine({ profile: "corpus" }); console.log(engine.normalize("a \\over b")); // \frac { a } { b } const reported = engine.normalizeWithReport("a \\over b"); console.log(reported.report.rewrite.iterations); // 2 ``` One shape is exempt from this convention: **`SyntaxNode` is not part of binding casing conversion.** It remains the single tree wire format across Rust serde, Python dictionaries, JavaScript objects, and JSON fixtures, so its field and variant names stay identical in every language. Everything else — reports, configs, lookup metadata — follows the snake_case / camelCase split above. The result is a consistent editing model across Rust, Python, and JavaScript, built on one core implementation rather than three hand-written copies. ## Source map [#source-map] Paths are relative to the [TeXForm repository](https://github.com/texform-dev/texform) root. They move as the code moves, so treat them as entry points rather than citations. | Path | What it defines | | --------------------------------------------- | ----------------------------------------------------------------------------- | | `ARCHITECTURE.md` | Crate boundaries, the pipeline, and the tree invariants this page summarizes. | | `crates/texform-core/src/document.rs` | The `Document` DOM, `NodeRef` reads, and the fallible editing surface. | | `crates/texform-core/src/ast.rs` | The internal arena and its panic contract. | | `crates/texform-interface/src/syntax_node.rs` | The `SyntaxNode` wire format and its serde derives. | | `crates/texform-core/src/serialize.rs` | The serializer behind `to_latex` and `to_tokenized_latex`. | | `crates/texform-wasm/Cargo.toml` | The WebAssembly binding's dependency surface — the facade only. | | `crates/texform-python/src/` | The Python binding's handle model and exception mapping. | ## Related [#related] - [Concepts: Pipeline](https://texform.dev/docs/pipeline) — The same journey, at a glance. - [Concepts: Parsing](https://texform.dev/docs/parsing) — Parse-result states and the Document editing model. - [Rules and the transform contract](https://texform.dev/docs/rule-contract) — How normalization is structured and enforced. - [Python API overview](https://texform.dev/api/python) — snake_case conventions and dict shapes in full. - [JavaScript API overview](https://texform.dev/api/js) — camelCase conventions and the WASM load model. --- # The knowledge base What the four kinds of knowledge-base record contain, where entries come from, how rendering validates them, and how to report a wrong argspec. https://texform.dev/docs/knowledge-base Read this if you want to know where the parser's knowledge comes from — what each record holds, how an entry's argument structure is decided, how it is verified before shipping, and how to report one that is wrong. The knowledge base is the layer that makes parsing more than string matching: for every name it supports, it records which package the name belongs to, which modes it is legal in, and what argument shape it takes. The parser consults it to decide how many arguments `\frac` consumes or whether `\sqrt` accepts an optional root. This page describes how those records are built and kept honest. ## The four kinds of record [#the-four-kinds-of-record] A package specification holds four kinds of record, and each answers a different question for the parser. | Record kind | What it describes | | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Commands | Control sequences such as `\frac` or `\operatorname`: their argument signature, the modes they are legal in, and how they attach to their arguments (prefix, infix, or declarative). | | Environments | `\begin{…}` / `\end{…}` pairs such as `align`, including any arguments the opening takes. | | Characters | Symbol entries such as `\alpha` or `\leq`: the Unicode character they stand for and the mode they belong to. | | Delimiters | Openers, closers, and the control names such as `\left` and `\right` that size them. | Commands and environments are the entries you see when you ask a package what it contains, and they are the ones that carry an [argspec](https://texform.dev/docs/argspec) — an xparse-style signature such as `m m` (two mandatory arguments) or `o m` (one optional, one mandatory). Characters and delimiters have no arguments to describe; they are what the [symbol reference](https://texform.dev/latex/symbols) is generated from. Records live in per-package specification files, one file per package, and are compiled into static tables when TeXForm is built, so the data the parser uses is the same data this site's references are generated from. ## Which packages are loaded [#which-packages-are-loaded] The built-in packages are `base`, `ams`, `physics`, `textmacros`, `bboldx`, `boldsymbol`, `color`, and `braket`. A parser or transform engine built without an explicit package list loads everything except `physics`, which stays opt-in because it redefines bra-ket commands that `braket` provides and standard commands such as `\div`, `\Re`, and `\Im`. Ask for `physics` by name when you need it, and list the other packages you want in the same call: naming packages replaces the default set rather than adding to it. ## Where entries come from [#where-entries-come-from] Entries are not transcribed by hand from prose documentation. They are built in stages: * **Package definitions are analyzed.** The argument structure of each command is derived from how its package actually defines it, so the recorded shape reflects the real macro, not a documentation summary. * **Annotation is LLM-assisted.** A language model proposes the argspec, allowed modes, and metadata for each entry, turning raw definitions into structured candidates. * **A human reviews the decisions.** Proposed entries are reviewed and accepted, corrected, or rejected before they enter the shipped knowledge base. The machine proposes; a person decides. ## How entries are verified [#how-entries-are-verified] Every entry's argspec is checked by rendering, not by inspection. From the argspec, test cases are generated with a one-factor-at-a-time strategy and rendered against three real engines — MathJax, KaTeX, and XeTeX. If the argument forms an entry claims do not render the way the spec predicts, that shows up as a renderer gap rather than passing silently. The full procedure, the three engines, and the meaning of the resulting **full** / **partial** / **none** statuses are described in [renderer validation](https://texform.dev/docs/renderer-validation). ## Frequency from corpus statistics [#frequency-from-corpus-statistics] Entries are weighted by how common they are. Command and environment frequencies come from counting occurrences across large real-world LaTeX corpora, so the catalog can distinguish a command that appears in millions of formulas from one that is rare. This is what lets [renderer support](https://texform.dev/latex/renderers) rank gaps by how often a command actually shows up, rather than treating every entry as equally important. ## How entries reach the runtime [#how-entries-reach-the-runtime] The reviewed specification files are compiled into TeXForm at build time and travel with the released Rust, Python, and JavaScript packages. There is no separate data download and no network lookup at parse time; the knowledge a build ships with is fixed for that release, and updates arrive with new releases. That means the specification files are not a configuration surface — editing them requires rebuilding TeXForm from source. To teach a running parser about a command it does not know, register it yourself with a context item, as [injecting custom commands](https://texform.dev/docs/inject-custom-commands) shows. To narrow what a parser accepts, drop names from the knowledge it loads instead of adding to it. ## Reporting an incorrect argspec [#reporting-an-incorrect-argspec] If a command parses with the wrong argument structure — too many or too few arguments, a missing optional argument, a wrong mode — it is almost always a knowledge-base entry to fix rather than a parser bug. Open an issue on the [TeXForm repository](https://github.com/texform-dev/texform) with the command name, the input you parsed, what you expected, and what TeXForm produced. A minimal reproducing formula is the most useful thing you can include. ## Source map [#source-map] Paths are relative to the [TeXForm repository](https://github.com/texform-dev/texform) root. They move as the code moves, so treat them as entry points rather than citations. | Path | What it defines | | ------------------------------------------- | ----------------------------------------------------------------------------------------- | | `crates/texform-knowledge/resources/specs/` | The per-package YAML specifications: one file per package, holding all four record kinds. | | `crates/texform-knowledge/build.rs` | The build step that compiles those YAML files into static tables. | | `crates/texform-knowledge/README.md` | The crate's own summary of record kinds and built-in packages. | | `crates/texform-core/src/knowledge.rs` | Which packages load by default, and how a parser's knowledge is assembled. | ## Related [#related] - [Renderer validation](https://texform.dev/docs/renderer-validation) — How rendering turns an argspec into a full, partial, or none status. - [Renderer support](https://texform.dev/latex/renderers) — Per-entry support across MathJax, KaTeX, and XeTeX. - [Symbol reference](https://texform.dev/latex/symbols) — The character and delimiter records, browsable. - [Reading argspecs](https://texform.dev/docs/argspec) — The signature notation used by command and environment records. --- # Renderer validation How TeXForm derives full, partial, and none renderer support: case generation, the three engines tested, and why partial happens. https://texform.dev/docs/renderer-validation Read this if you want to know what the **full**, **partial**, and **none** labels on [renderer support](https://texform.dev/latex/renderers) actually mean — how they are measured, against which engines, and why a command ends up **partial** rather than **full**. ## How support statuses are determined [#how-support-statuses-are-determined] Support is measured by rendering, not by reading documentation. Each entry in the [knowledge base](https://texform.dev/docs/knowledge-base) carries an [argspec](https://texform.dev/docs/argspec) describing its argument structure. From that argspec, a set of test cases is generated and each case is rendered against three engines. The status for a renderer summarizes the outcome of an entry's **positive** cases — cases that are expected to render: * **full** — every positive case rendered without error. * **partial** — at least one positive case rendered and at least one failed. * **none** — no positive case rendered, or the renderer does not recognize the entry. Test cases are generated with a one-factor-at-a-time (OFAT) strategy: from each argspec, one case fills the required arguments and omits the optional ones, then further cases vary a single argument form at a time — enabling one optional argument, dropping the braces around a required argument, and so on. Varying one factor per case means a failure points at a specific argument form rather than a tangle of interacting ones. This generation and three-renderer comparison is performed by `argspecs-validator`, an internal tool that is not open source. It reads the package specs, generates the OFAT cases, runs them through the three renderers, and writes the pass/fail results that this site's support data is built from. ## The renderers tested [#the-renderers-tested] Three engines are exercised, chosen to span the common deployment targets for LaTeX math: * **MathJax** (version 4) — the reference web renderer, with the package matching each entry enabled. * **KaTeX** (version 0.16) — a faster web renderer with a smaller built-in command set and no per-package isolation, so its built-in commands are always active. * **XeTeX** (TeX Live 2026) — a full TeX engine, run with only the packages mapped to the entry under test. Validation is package-scoped: each entry is rendered against its own package's behavior, not against a runtime with every package loaded at once. A result therefore answers "does this entry match this package's behavior in this engine?" — not "what happens when every package is enabled together?" ## Why "partial" happens [#why-partial-happens] A **partial** status means an entry works in one argument form but not another. The OFAT cases surface this because they probe each form independently. Common causes: * **Bare-argument forms.** A renderer may accept `\rule{1pt}{1pt}` but reject the unbraced `\rule 1pt{1pt}`, so the braced cases pass and the bare case fails. * **Optional-argument forms.** An optional argument that one engine does not implement fails only on the cases that exercise it, leaving the baseline case passing. * **Spacing and box commands.** Length- or box-valued arguments are a frequent source of divergence between engines. Negative cases — inputs an entry should reject, such as text injected into a math-only slot — are tracked separately and do not contribute to the support status. ## How XeTeX gaps should be read [#how-xetex-gaps-should-be-read] Because XeTeX is validated in math mode with only the entry's own package mapped, some commands report **none** for XeTeX even though they render in a complete XeTeX document. Text-mode commands and commands that depend on packages outside the one under test fall into this category. The XeTeX column is a faithful report of the validation scope, not a claim that XeTeX cannot render the command in any context. ## Renderer versions and refresh [#renderer-versions-and-refresh] The current data was produced with MathJax 4, KaTeX 0.16, and XeTeX from TeX Live 2026. The results are regenerated when the knowledge base or the validation tooling changes; the support data on this site reflects the most recent run rather than a fixed schedule. ## Source map [#source-map] The `argspecs-validator` that generates and runs the OFAT cases is not open source, so there is no public path for the generator, the renderer harnesses, or the raw result files. What is public is the input it reads, in the [TeXForm repository](https://github.com/texform-dev/texform): | Path | What it defines | | ------------------------------------------- | --------------------------------------------------------------------------------------------------- | | `crates/texform-knowledge/resources/specs/` | The per-package argspecs that every generated test case is derived from. | | `crates/texform-argspec/` | The signature language those argspecs are written in, including which argument forms exist to vary. | ## Related [#related] - [Renderer support](https://texform.dev/latex/renderers) — The full, partial, and none statuses, per entry and engine. - [The knowledge base](https://texform.dev/docs/knowledge-base) — Where the argspecs being validated come from. - [Reading argspecs](https://texform.dev/docs/argspec) — The notation that decides which cases get generated. --- # Argspec design Why TeXForm describes command signatures with an xparse-style notation, what it extends, and which signatures it cannot express. https://texform.dev/docs/argspec-design Read this if you want to understand why TeXForm describes argument signatures the way it does — why an xparse-style string rather than a structured table, what was added to xparse and why, and where the notation runs out of expressive power. For the notation itself and how to write one, start with [Reading argspecs](https://texform.dev/docs/argspec); this page is about the reasoning and the limits. ## Why a signature notation at all [#why-a-signature-notation-at-all] The parser is not purely syntactic. To turn `\frac{a}{b}` into a syntax tree it has to know that `\frac` takes two mandatory arguments, and to turn `\sqrt[3]{x}` into the right shape it has to know that `\sqrt` takes one optional argument before its mandatory one. That argument shape is not recoverable from the surface syntax — `\frac x y` and `\frac{x}{y}` describe the same node, and a command that took one argument would bind them differently. So every known command and environment in the knowledge base carries a description of its argument structure, and the parser consults it while reading. The question is what form that description takes. A structured table — one row per argument, with explicit fields — is unambiguous but verbose: the vast majority of commands take only mandatory and optional arguments, and writing five lines of structured data to say "two mandatory arguments" buries the signal. A signature is read and audited far more often than it is written, so the format optimizes for being scannable at a glance. ## Why xparse-style notation [#why-xparse-style-notation] LaTeX3's [xparse](https://ctan.org/pkg/xparse) package already solved the same problem for document-level commands: it encodes a full argument skeleton as a short string of single-letter tokens. `m m` is two mandatory arguments; `o m` is optional-then-mandatory; `s o m` is a star, an optional, and a mandatory. The notation is compact, the letters are mnemonic, and it is already familiar to anyone who has defined a LaTeX command. TeXForm borrows that convention because it inherits those properties for free. The argspec string for a command reads as the command's shape — `s m g` is visibly "optional star, mandatory argument, optional trailing group" — without a reader having to expand a table in their head. The base letters `m`, `o`, `s`, `g`, `r`, and `d` keep their xparse meanings, so the learning cost is mostly already paid. This notation is xparse-inspired, not xparse-compatible. It is TeXForm's own signature language, parsed by TeXForm's own parser, and it diverges from xparse wherever describing math commands calls for it. The next sections cover where. ## What TeXForm adds to xparse [#what-texform-adds-to-xparse] xparse describes *how* an argument is delimited, but not *what kind of value* it holds, because in LaTeX3 the value's interpretation is the macro's own business. A math parser does not have that luxury: it has to know whether an argument is a delimiter, a length, an integer, a name, or a column template, because each is read and validated differently. So TeXForm adds the following, each driven by a concrete need from the supported packages: * **Value-kind annotations** (`:T`, `:O`, `:D`, `:N`, `:L`, `:I`, `:K`, `:C`). A `:` suffix declares how a slot's content is interpreted — text, operator-name content, delimiter, control-sequence name, dimension, integer, key–value, or column template — defaulting to math content when absent. `\label{sec:intro}` is `m:N`, because its argument is a name string that must reject escape sequences, not a math subtree; `\operatorname` is `s m:O`, because its name body parses as math content but must serialize as a compact operator-name run. * **The nullable modifier** (`?`). Appended to an annotation, it marks a slot that is present but may carry an empty value, distinct from an optional slot that may be absent. `\genfrac`'s leading delimiter and thickness arguments are `m:D? m:D? m:L?`: each must be written, but each may be empty (`{}`), meaning "no delimiter" or "default rule thickness". * **The required braced group** (`m{}`). xparse's `g` reads an optional `{...}`; `m{}` is its required counterpart. The distinction from `m` is that the braces are structural grouping, not part of the value, which matters when the argument is later serialized. * **Delimiter-choice pair lists** (`r<…>`, `d<…>`). A plain `r` or `d` names one delimiter pair. The `` list lets a single argument slot accept several delimiter shapes at once. That last one is worth its own section, because it is the extension with the most design weight behind it. ## The delimiter-choice problem [#the-delimiter-choice-problem] Several `physics` commands wrap a single argument in any of several delimiter shapes. `\qty` accepts `(x)`, `[x]`, `{x}`, or `|x|`, and all four mean the same thing: one argument, differently fenced. A plain xparse `d` or `r` can name only one pair, so the only xparse-shaped way to cover four shapes is four separate optional slots — and that is wrong. Four slots tell the consumer there are four independent arguments, when there is really one argument that can arrive in four shapes. A position-indexed reader cannot tell which slot the value landed in. The pair-list form fixes this by mapping every listed shape to the *same* slot. `d<(,)><[,]><{,}><|,|>` is one optional argument; the parser tries each pair in order and reads with the first that matches, recording which pair won so the original fencing can be restored on serialization. This is genuinely outside xparse's vocabulary, and it is the reason the notation is described as xparse-*style* rather than an xparse subset. ## Known expressive limits [#known-expressive-limits] An honest signature notation is one whose boundaries are documented. The following shapes cannot be expressed exactly today. None of them block parsing — each supported command has a working best-fit signature — but in these cases the signature is an approximation, and that is worth knowing if you are auditing behavior or writing your own. ### Value kinds are coarse [#value-kinds-are-coarse] The value-kind annotations name broad categories, not specific accepted-value sets. Two annotations carry most of this weight and both are deliberately wide: * `:N` covers any name-like argument but cannot narrow to a specific set of accepted literals. * `:C` describes a full column template (`c|c`, `p{1em}`, `@{...}`) and cannot be restricted to, say, a short list of alignment letters. So a command that accepts only a small fixed set of literal tokens still has to be modeled as `:N` or `:C`, which over-accepts: the signature says "a name" or "a column template" where the command really means "one of these few values". Commands whose true argument is a restricted enumeration — among them `\breakAlign`, `\mmlToken`, `\parbox`, and `\cfrac` — fall into this gap. They parse correctly, but their argspec describes a wider value space than the command actually accepts. ### One slot cannot mix visible delimiters with structural braces [#one-slot-cannot-mix-visible-delimiters-with-structural-braces] The pair-list form treats every matched `` pair as a *visible* delimiter — a fence that is part of the value and should survive serialization. That is the right model for `\qty`, where `\qty{x}` is meant to render with braces shown. It is the wrong model for the `physics` matrix-quantity family, where the brace shape means something different from the others. For `\mqty`, the forms `\mqty(a)`, `\mqty[a]`, and `\mqty|a|` produce visible delimiters, but `\mqty{a}` uses the braces only as input grouping and renders with no outer delimiter at all. The closest single-slot signature is `s d<(,)><[,]><{,}><|,|>`, which correctly keeps all four shapes in one argument slot — but it forces the `{,}` branch to be treated as a visible `Paired` delimiter, so the signature cannot say "this branch triggers a read, but the braces themselves are not shown". Splitting the brace branch out as a separate `g` slot would express the structural grouping, but at the cost of breaking the single-slot guarantee the pair list exists to provide. The affected `physics` families include `MatrixQuantity` (`\mqty` and friends), `Eval`, `Differential`, and parts of the vector-operator family. Where a brace branch genuinely fences a visible delimiter — `\qty`, `\Re`, `\Im` — the pair-list model is exact. ### Some MathJax behaviors are intentionally not modeled [#some-mathjax-behaviors-are-intentionally-not-modeled] A few behaviors in MathJax's own command handling are out of scope for a static signature, by design rather than by gap: * **Stack-item operators.** The vector-operator family (`\grad`, `\curl`, `\div`, `\laplacian`, …) binds a following fenced group through MathJax's stack-item machinery, not by reading an argument. Their argspec is empty (`""`); the fence association is a transform concern, not a parse-time argument. * **Bare-token reads.** MathJax's `\dd x` can swallow a following bare token as an argument. TeXForm does not model this: `\dd x` parses as `\dd` followed by a separate `x`. * **Value-driven branching.** A signature cannot say "if the first argument is `\frac`, read another argument", so behaviors where one argument's content changes how later arguments are read are not expressible. These are exclusions, not failures: the signature describes a fixed argument skeleton, and behaviors that depend on token lookahead beyond delimiter matching or on the content of earlier arguments fall outside what a skeleton can carry. ## Validation is part of the notation [#validation-is-part-of-the-notation] Because signatures are written by hand — in the knowledge base and, for custom commands, by users — the parser that reads them rejects malformed strings precisely rather than guessing. Invalid combinations are caught at the point they occur: `!` on a required slot, `?` on a star or column, an unknown annotation letter, `m{x}` (only `m{}` is allowed), a value-kind annotation on `s`. The same parser backs both the compile-time macro that builds the built-in knowledge base and the runtime `validate_argspec` function, so a string that validates at runtime is read identically to one baked in at build time. That shared path is what makes [Reading argspecs](https://texform.dev/docs/argspec) a reference you can trust against the actual parser. ## Related [#related] * [Reading argspecs](https://texform.dev/docs/argspec) — the notation reference, with one real command per parameter kind. * [Inject custom commands](https://texform.dev/docs/inject-custom-commands) — writing an argspec and teaching it to the parser. * [The knowledge base](https://texform.dev/docs/knowledge-base) — how the signatures for built-in commands are built and validated. --- # Rules and the transform contract The rule metadata model, fixed-point scheduling, the eliminated-forms contract, and how to read rule source. https://texform.dev/docs/rule-contract Read this if you want to contribute a rewrite rule or audit why the transform engine produces a given output. For the user-facing view of profiles and the phase pipeline, start with [Concepts: Transforms](https://texform.dev/docs/transforms). ## The rule metadata model [#the-rule-metadata-model] Normalization is not find-and-replace. Each rewrite rule declares the metadata the engine uses to schedule rules, verify convergence, and enforce its output contract: * **`key`** — a stable `package/name` identifier used in transform reports and build-time filters. * **`enabled_by_packages`** — the packages whose presence makes the rule loadable. * **`level`** — the first profile that accepts the rule output: `Authoring`, `Faithful`, `Corpus`, or `Equiv`. * **`summary`** — a one-line description. * **`fidelity`** — the worst-case equivalence guarantee over the declared input domain: `Math`, `Reading`, or `Render`. * **`triggers`** — the targets the scheduler watches. * **`consumes`** — forms the rule eliminates or may touch. * **`produces`** — forms the rule may introduce. ### Level and fidelity are orthogonal [#level-and-fidelity-are-orthogonal] A rule's level classifies the intended product, not the size of its visual change: | Level | Intent | | ----------- | ---------------------------------------------------------------------------------------------------------------------------- | | `Authoring` | Author-editable canonical syntax that preserves reasonable notation and style choices. | | `Faithful` | Render-faithful universal forms for compact, package-specific, or legacy input. | | `Corpus` | Complete, stable canonical forms suitable as training labels; reading-identical layout and vocabulary variants may collapse. | | `Equiv` | Comparison-only intermediates that need not remain suitable as source or training labels. | Fidelity is a separate worst-case guarantee ordered `Math < Reading < Render`. `Render` preserves rendering under the reference renderer, `Reading` preserves notation content, reading order, and structural roles while allowing layout changes, and `Math` preserves mathematical meaning over the declared domain while allowing notation and rendering to change. Authoring, Faithful, and Corpus require at least Reading; Equiv requires at least Math. A stronger fidelity does not automatically move a rule to an earlier level. Because the levels are cumulative, the Equiv profile runs every Corpus rule and then the `Equiv` rules on top. Those rules drop distinctions that two spellings of the same mathematics do not need to agree on: `\cfrac{a}{b}` becomes `\frac{a}{b}`, `\widehat` collapses onto `\hat`, explicit `\big` / `\Big` delimiter sizing is dropped, and fixed horizontal spacing such as `\quad` disappears. That is exactly why they are comparison-only — `\cfrac` was chosen by an author for a reason, and an Equiv output is not a faithful replacement for the source. ## Fixed-point scheduling [#fixed-point-scheduling] Rewrite is one phase in a fixed sequence — pre-rewrite LowerAttributes, Rewrite, post-rewrite LowerAttributes, FinalizeAst, FlattenGroups, then FinalizeAst a second time. The second FinalizeAst pass is the same idempotent pass, re-run because flattening can bring nodes next to each other that were not adjacent before; it is skipped when FlattenGroups is disabled, since nothing new can become adjacent. FinalizeAst is the last phase that mutates the tree, and the eliminated-forms check that follows is read-only. When a transform context is built, the engine filters all registered rules through the selected profile's levels, build-time filters, and enabled packages. At run time it repeatedly applies that precompiled plan until a full pass changes nothing or `max_iterations` is reached. The transform report exposes this directly: `rewrite.iterations` is the fixed-point iteration count, including the final unchanged check, and `rewrite.rules` lists each recorded rule with its `key`, `applied_count`, and `skipped_count`. Plain `normalize` returns only the string. Request the report with `normalize_with_report` in Python or `normalizeWithReport` in JavaScript. ## The eliminated-forms contract [#the-eliminated-forms-contract] A rule that removes a form declares it in `consumes.eliminates`. After the complete pipeline has run, the engine checks that no declared eliminated form remains. A violation is a transform error, not a warning or a half-normalized result. The `texform-regression` corpus harness runs the same contract over real-world data. Known, reasoned exceptions live in `regression/contract_exceptions.yaml`; a new unlisted violation means the declared contract and actual behavior no longer agree. ## What callers can change [#what-callers-can-change] The rule set is compiled into the engine, so there is no run-time registration hook: you cannot add a rule of your own without building TeXForm from source. What you can do from the outside is subtract. Pass `disable_rules` with the keys you want left out when you construct the engine — `disable_rules=["physics/dv-to-frac-d"]` in Python, `disableRules` in JavaScript — and the plan is compiled without them. [Disable a rule](https://texform.dev/docs/disable-a-rule) walks through finding the key in a transform report and turning it off. ## How to read rule source [#how-to-read-rule-source] Rules live under [`src/rewrite/rules/`](https://github.com/texform-dev/texform/tree/main/crates/texform-transform/src/rewrite/rules) using `///.rs`. The level directory is one of `authoring`, `faithful`, `corpus`, or `equiv`; the build script verifies it agrees with `RuleMeta.level`. Rule keys remain `package/name` regardless of the source path. Most rules use `define_rule!`, which keeps metadata beside the rewrite logic: ```rust define_rule! { pub static OVER_TO_FRAC: OverToFracRule { key: Base / "over-to-frac", level: Authoring, summary: "Rewrite infix over to an explicit frac command.", fidelity: Render, enabled_by_packages: [Base], triggers: cmd_targets![&base::cmd::OVER], consumes: RuleConsumes { eliminates: cmd_targets![&base::cmd::OVER], touches: &[], }, produces: RuleProduces { targets: cmd_targets![&base::cmd::FRAC], }, apply(rule, cx, node_id) { // match the \over infix and rebuild it as \frac{left}{right} } } } ``` This rule eliminates `\over`, produces `\frac`, first enters at Authoring, and guarantees Render fidelity. Other useful examples include alias rules such as `physics/derivative-to-dv`, which eliminates `\derivative` and produces `\dv`, and Corpus cleanup rules such as `base/goodbreak-drop`. The former `displaylines-to-gather-env` proposal was rejected and removed because it could not guarantee Reading over a reasonable declared domain. ## Source map [#source-map] Paths are relative to the [TeXForm repository](https://github.com/texform-dev/texform) root. They move as the code moves, so treat them as entry points rather than citations. | Path | What it defines | | ------------------------------------------------------ | -------------------------------------------------------------------------------------------- | | `crates/texform-transform/src/engine.rs` | The fixed phase order, including the second FinalizeAst pass and the eliminated-forms check. | | `crates/texform-transform/README.md` | Phase behavior, profile-to-level mapping, and the fidelity definitions. | | `crates/texform-transform/src/rewrite/rules/README.md` | The rule authoring guide and the metadata contract. | | `crates/texform-transform/src/rewrite/rules/` | Every built-in rule, laid out as `///.rs`. | | `regression/contract_exceptions.yaml` | The reasoned exceptions to the eliminated-forms contract over corpus data. | ## Related [#related] - [Concepts: Transforms](https://texform.dev/docs/transforms) — Profiles, the phase pipeline, and transform reports. - [Disable a rewrite rule](https://texform.dev/docs/disable-a-rule) — Find a rule key in a report and switch that rule off. - [Internals: Architecture](https://texform.dev/docs/architecture) — Where the transform engine sits among the crates. - [Rule source on GitHub](https://github.com/texform-dev/texform/tree/main/crates/texform-transform/src/rewrite/rules) — The complete rule tree and authoring guide. --- # Changelog Notable user-visible changes in each TeXForm release, covering the Rust, Python, and JavaScript channels. https://texform.dev/docs/changelog TeXForm keeps a single changelog for all three release channels — one version number covers the Rust crate, the Python package, and the JavaScript package. See [Stability and versioning](https://texform.dev/docs/stability-and-versioning) for what each release type may contain. The changelog is maintained in the repository and follows the [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) format. It is rendered below from [`CHANGELOG.md`](https://github.com/texform-dev/texform/blob/main/CHANGELOG.md) at build time. Treat the changelog as the release timeline. For the current shape of a configuration object or a report, [Transforms](https://texform.dev/docs/transforms) and the [API reference](https://texform.dev/api) are the authority. ## 0.10.1 · 28 Oct 2026 This release lets the PyPI package install the `texform` command-line tool, and fixes transform failures and output that did not reparse, mostly with `physics`. One rule key changes; see **Changed**. ### Changed - The rule key `base/operatorname-to-named-function` becomes `ams/operatorname-to-named-function`, because the rule only applies when `ams` provides `\operatorname`. Configurations that enable or disable the old key must use the new one; the old key is not kept as an alias. Output is unchanged. ### Added - The PyPI package installs the `texform` command, also available as `python -m texform`, so the command-line tool no longer needs a Rust toolchain. Install it with `uv tool install texform`, or run it once with `uvx texform`. It runs the same code as `cargo install texform-cli`, with the same commands, output, and exit statuses; each invocation adds about 10 ms of Python startup, and `--lines` and `serve` throughput is unchanged. ### Fixed - With `physics`, `\eval`, `\eval*`, and `\evaluated` without a following `(`, `[`, or brace group no longer fail with a rewrite contract violation. As in MathJax, they expand to `\left.\vphantom{\int}\right|`, so `\eval x` becomes `\left.\vphantom{\int}\right| x`. - With `physics`, an ordered derivative whose variable already has a superscript no longer produces a double superscript that fails to reparse: `\dv[2]{f}{x^2}` now gives `\mathrm{d}{x^{2}}^{2}` in the denominator. `\pdv` and `\fdv` are fixed the same way. - A delimited argument whose content ends with an unpaired copy of its opening delimiter is kept in a protective brace group, and the parser strips that group again. Under Equiv with `physics`, `\qty(a{\big(}b)` now becomes `\qty({a(b})` instead of `\qty(a(b)`, which failed to parse. - Rewriting a scripted `\break` or `\newline` attaches its scripts to an empty base, so `\break'` becomes `\\{}'`, and FlattenGroups keeps the group in `{\cr}_1`. Both outputs previously changed when parsed again. - Dropping `\goodbreak` used as a script base no longer panics in Corpus and lower profiles: `\goodbreak^2` becomes `{}^{2}`. ## 0.10.0 · 14 Oct 2026 This release replaces the serializer's seven option axes with two output styles: `spaced`, the previous default, and a new `compact` style for human-readable LaTeX. With `physics`, named functions such as `\sin` and `\exp` now parse the argument that physics.sty prints as an auto-sized fence, and three Faithful rules make that fence explicit. Normalization also no longer removes a group or a delimiter size that kept a preceding command from reading the next token as an argument on reparse. The serializer change is breaking; see **Changed** for migration notes. ### Changed - **Breaking:** `SerializeOptions` now holds a single `style: Style`, either `Style::Spaced` (default) or `Style::Compact`. `CommandSpacing`, `MathGroupInnerSpacing`, `AdjacentCharSpacing`, `ScriptSpacing`, `ScriptOrder`, `InfixGrouping`, and `EnvironmentNameSpacing` are removed, and Python `to_latex` and JavaScript `toLatex` accept only the `style` key and reject the removed ones. Spaced output is byte-for-byte identical to the previous default and stays byte-stable across releases. Replace axis settings with `SerializeOptions { style: Style::Compact }`, `to_latex(style="compact")`, or `toLatex({ style: "compact" })`; combinations that compacted only some axes, `sup_first` script order, and always-explicit infix operands have no replacement. - With `physics`, the 26 `\trigbraces` names (`\sin` … `\coth`, `\asin` … `\acot`, `\log`, `\ln`) take `o d()`, and `\exp`, `\det`, `\Pr`, `\tr`, and `\Tr` take one paired `d<(,)><[,]><{,}>` slot, because physics.sty prints that argument as an auto-sized fence. A delimiter separated from the command by a group is now literal, and FlattenGroups keeps that group, as in `{\cos}(x)` or `\cos{}(x)`. An unclosed parenthesis such as `\sin(x` is a parse error with `physics`, as in LaTeX; across the regression datasets, 144 more formulas fail to parse with `physics` loaded. ### Added - Compact output keeps only lexically required spaces and omits braces around a script that is a single character, a single prime, `~`, or a known command without argument slots, adding a readability space before a following ASCII letter or digit (`\frac{a}{b}+x_i^2`, `x^2 3`). Scripts holding a command with argument slots, an unknown command, or a declarative switch keep their braces, as in `x^{\sin}[2]` with `physics` or `x^{\rm}a`. Compact output preserves the parsed structure and is text-idempotent; its layout may improve across releases. - `texform normalize --style spaced|compact` applies to text, `--json`, and `--report` output, and the `serve` `configure` request accepts an optional `style` and echoes it in `resolved`. `tokenize` always uses spaced output. - Faithful rules expand physics named functions for renderers without physics. `physics/named-fn-paren-expand` rewrites `\sin(x)` and `\sin[n](x)`, and the same forms of the other `\trigbraces` names that have base commands, to `\sin\left(x\right)` and `\sin^{n}\left(x\right)`; `\sin[n]` without parentheses is unchanged. `physics/physics-operator-fence-expand` fences the argument of `\exp`, `\det`, and `\Pr`. `physics/named-fn-to-operatorname` rewrites physics-only names such as `\asin`, `\csch`, `\tr`, `\erf`, and `\Trace` to `\operatorname`, with the same expansion where the name takes an argument. The explicit fence is an inner atom, so it can add up to 1/6 em on each side; the rules declare `Reading` fidelity. ### Fixed - FlattenGroups keeps a group when splicing it would let a known command read the following token as an omitted trailing optional, star, group, or delimited argument. With `physics`, `\sin{[2]}x`, `{\sin}[2]x`, and `\sin{}[2]x` no longer become `\sin[2]x`, and `{\qty}(a)` no longer becomes `\qty(a)`; `\text{a{\\}*b}` no longer becomes `\text{a\\*b}`. The protection applies in every profile, and reports count it as `guard_hits.argument_capture` (`argumentCapture` in JavaScript). - Equiv rules that drop a fixed delimiter size no longer expose the delimiter to the preceding command: `a\\\big[x\big]` becomes `a\\{}[x]` instead of `a\\[x]`, which reparsed `x` as the line-break dimension, and with `physics` `\ln\Bigr[x\Bigr]` no longer becomes `\ln[x]`. - The physics authoring aliases `sine-to-sin`, `cosine-to-cos`, `naturallogarithm-to-ln`, `probability-to-pr`, `trace-to-tr`, and `trace-capital-to-Tr` rename only a command not followed by `(`, `[`, or a brace group. `\trace(\rho)` and `\Probability(\frac{x}{y})` no longer gain an auto-sized fence, and `\sine(x)` no longer fails the conformance check in `authoring` or stops later rules in `faithful`. - With `abort_on_error`, a failure inside a command argument is reported at its source: `a+\frac{1}`, `a+\hspace{x}`, and `a+\text{$x}` no longer point at `a`, and `array` and `align` are no longer reported as unknown or as not allowed in text mode. An unclosed inline `$` is reported at its opener in strict and non-strict parsing. Which inputs fail is unchanged. - Trees built with `from_syntax` keep a space after a control-word delimiter such as `\langle`, or an error snippet ending in a control word, before an ASCII letter, and after a math-mode control sequence before a sibling `[` or `*`. Serialized output of parsed input is unchanged. ## 0.9.1 · 9 Oct 2026 This release adds the `color` package to the default knowledge base, gives transform reports warnings for unknown input, and adds a rule catalog API and an explicit `texform/web` entry for browsers. Unknown commands now keep the brace groups that follow them, so their argument boundaries survive normalization. Formulas that use color commands now parse with known signatures, which changes their parsed trees and normalized output. ### Added - The `color` package covers `\color`, `\textcolor`, `\definecolor`, `\colorbox`, and `\fcolorbox`, and is loaded by default in the library and the command-line tool, which now default to seven packages. `\color[model]{color}` is a declaration with LaTeX and MathJax 3/4 semantics, scoped to its enclosing group, not the two-argument form of MathJax's legacy `colorv2`. `\textcolor` takes `o:N m:N m` in math mode and `o:N m:N m:T` in text mode. Color names, models, and specifications are kept as literal strings: they are not validated and control sequences in them are not expanded, so `\color{\mycolor}x` still fails to parse. - Transform reports include `warnings` that list the unknown command and environment names in the input, deduplicated and sorted by kind and name. They are collected whenever a report is requested, even with every phase disabled, and do not make a transform fail. Rust, Python, and JavaScript report types expose them, and `texform serve` `normalize` accepts an optional `report` request that returns the same report; plain responses and normalizer protocol v1 are unchanged. - `texform::list_rules()`, Python `list_rules()`, and JavaScript `listRules()` return every built-in rewrite rule sorted by key, with its `level`, `fidelity`, `summary`, and enabling packages, so rule keys for `disableRules` and report statistics can be discovered. The facade re-exports `RuleLevel` and `RuleFidelity`; TypeScript adds `RuleFidelity` and `RuleInfo`, and the Python stubs add `RuleLevel`, `RuleFidelity`, and `RuleInfo`. - The npm package adds a `texform/web` entry for browsers that need to defer WebAssembly loading or supply the module themselves. Call `init()` or `init({ wasm })` before using the API; it is idempotent, can be retried after a failure, and reports typed initialization errors. The default bundler entry is unchanged. With Vite, exclude `texform` from `optimizeDeps`. ### Fixed - Brace groups directly after an unknown command are no longer flattened. In `\unknown{a}{}{bc}`, the empty group and the braces around `bc` were treated as redundant and removed, which lost the argument boundaries. Every profile now keeps them, including empty groups and a group used as a script base such as `\unknown{a}^2`, and the guard report counts them as `unknown_command_arguments`. This keeps the boundaries, not the meaning of unknown input; other phases still normalize the group contents. ## 0.9.0 · 4 Oct 2026 This release is a round of fixes that make TeXForm safer to run on real-world input. Parsing no longer panics on any source text, reports each problem with its specific diagnostic kind at its actual source position, and reads several plain TeX constructs as TeX does instead of silently misparsing them. Input that TeXForm cannot read correctly is now rejected with a diagnostic rather than turned into a wrong tree. The `Corpus` profile no longer drops modifiers whose visible effect depends on the math style, so its output keeps the rendering of its input. To parse the plain TeX macros `\root…\of` and `\buildrel…\over` correctly, the release adds the `u{\name}` form to the ArgSpec syntax; its new argument-form variant is the only breaking change, for Rust code that matches argument forms exhaustively. ### Added - **Breaking:** The ArgSpec syntax gains the until form `u{\name}`. It describes plain TeX delimited parameters such as `\def\root#1\of#2`, whose argument runs up to a terminating control word instead of being one token or one brace group, which no existing form could express. The argument collects tokens up to the first terminator outside brace groups and strips one brace layer when it is exactly one group; empty content is valid, and a missing terminator is a parse error. The terminator must be a single control word such as `\of` or `\over`, and ArgSpec validation, including for custom entries, rejects characters, control symbols, and multi-token terminators. `ArgumentKind`, `ArgKindRef`, and `ArgSpecFormInfo` gain an `Until { close }` variant, and bindings, columnar export, and CLI output name the form `"until"`; exhaustive matches over argument forms must handle it. - The knowledge base covers LaTeX text boxes and related commands, so input such as `\text{\mbox{$x$}}`, `\raisebox{1ex}{$x$}`, or `\left\llbracket x\right\rrbracket` parses. `\raisebox` and `\scalebox` are recognized with their optional arguments; `\mbox`, `\fbox`, `\hbox`, `\makebox`, and `\parbox` work in text mode as well as math mode; `\label` and the `math` environment are allowed inside text containers; `\ensuremath` reads its argument as math in both modes; nested `\text` is allowed in text mode; `\emph` works in math mode; and `\llbracket` and `\rrbracket` are delimiters. These records describe parsing, not renderer support, and dimension arguments still require literal lengths. ### Changed - The ArgSpec of the command `root` changes from `m` to `u{\of} m`. With `m`, the parser read only one token or group as the degree and left `\of` and the radicand as sibling nodes, so unbraced degrees such as `\root n_i\of{x}` were misparsed and `\root\of{x}` was rejected. A parsed `root` now has a degree slot and a radicand slot, and a `\root` without `\of` is a parse error. Normalized output is unchanged for previously accepted input. - The ArgSpec of the command `buildrel` changes from `m` to `u{\over} m`. With `m`, `\buildrel` took only the first token of the annotation, and the parser used a source-text lookbehind to split the rest with an infix `\over`, so `a\over\buildrel b\over c` was rejected as an ambiguous infix and a braced `\over` inside the annotation was mistaken for the separator. A parsed `buildrel` now has an annotation slot that runs to the first `\over` outside braces and an operator slot, and a `\buildrel` without `\over` is a parse error. Scripts after the operator attach to the whole relation. The Faithful rule `base/buildrel-expand` is replaced by the Authoring rule `base/buildrel-to-stackrel`, which rewrites `\buildrel A\over B` to `\stackrel{A}{B}`; rule selections that name the old key should use the new one. Faithful, Corpus, and Equiv output is unchanged for previously accepted input, and Authoring now succeeds instead of failing its rewrite contract. - `Corpus` and lower profiles keep `\dfrac`, `\tfrac`, `\limits`, and `\nolimits`. Their visible effect depends on the math style, which a single formula does not know, so dropping them could change the fraction size or move the scripts. `Equiv` still rewrites them. ### Fixed - Source text containing a character the lexer cannot tokenize, such as an ASCII control character other than tab, newline, carriage return, or form feed, returns an `invalid-character` diagnostic instead of panicking. In Python the panic surfaced as `PanicException`, which escaped ordinary `except Exception` handlers; callers now receive a regular `ParseError`. - Diagnostics name the specific problem. Internal parser branch failures such as `not a command` no longer replace the real diagnostic, an environment left open at end of input reports `unclosed-environment` with `\end{name}` as the expected token, and an environment used in a disallowed mode reports `environment-mode-error` in both strict and lenient parsing. `ParseDiagnosticKind` in Python and TypeScript includes the new kinds. - Diagnostics point at the actual source position. With `reject_unknown`, `\frac{\bad}{x}` reported the unknown command at `\frac`, and `a+\bad` at `a`; spans inside argument content were also shifted after a comment or a run of spaces. Errors now keep the failing token's span, and diagnostics and node spans inside arguments map token by token back to the source. - Plain TeX alignment macros (`\matrix`, `\pmatrix`, `\cases`, `\eqalign`, `\eqalignno`) split rows at `\\` and `\newline` as well as `\cr`, and drop a final row terminator that has no `*` or spacing. `\eqalignno` no longer fails unless every row ends in `&(…)`: it becomes `align*`, and only rows with a numbering cell get a `\tag` (or `\tag*` when the cell is not wrapped in parentheses). - A script at the start of an alignment row attaches to an empty base in that row instead of to the preceding row separator: `\begin{matrix}a\\_i b\end{matrix}` no longer serializes as `\\ _{i} b`. - In `array`, `darray`, `eqnarray`, and `eqnarray*`, `\\ [-5pt]` reads the bracket as row spacing, as LaTeX does. amsmath environments still read it as row content, so a row can start with an interval such as `[a,b]`. - An unbraced `\left…\right` or `$…$` argument keeps its delimiters: `\frac\left(x\right)2` parses like `\frac{\left(x\right)}{2}` instead of `\frac{x}{2}`, and `\text$x$` like `\text{$x$}`. - A comment between a control word and following letters inside an argument no longer merges them: `\sqrt[\le% comment` followed by `q]{x}` on the next line was read as `\sqrt[\leq]{x}`. - Custom signatures with delimiters such as `d**` or `r~~` match `*`, `$`, `&`, `#`, `^`, `_`, and `~` in the input instead of always treating the argument as absent. ArgSpec validation rejects delimiters that can never match: `%`, `'`, `’`, and ASCII control characters. - `\hspace*{…}` parses in math and text modes. - A text environment whose body ends with a group, inline math, or whitespace before `\end` no longer reports a spurious diagnostic. ## 0.8.0 · 30 Sep 2026 This release makes `Document` a complete API for building and editing formulas. Every node kind now has a constructor, so a formula can be built from scratch as a tree and serialized, and every construction and edit is checked against the document's knowledge base, so the result always conforms to it, as parser output does, and transforms accept it. Deterministic edits, rooted node paths, cross-document imports, and a columnar export cover targeted edits and bulk analysis. To support this, `KnowledgeBase` becomes an immutable object shared by parsers, engines, and documents. The release also fixes script, text-argument, and declaration-scope handling in alignments and text mode. The knowledge base and construction changes are breaking; see **Changed** for migration notes. ### Changed - **Breaking:** Package selection, custom items, and removals move from parser and engine builders to `KnowledgeBaseBuilder`, and knowledge queries move from `Parser` and `TransformEngine` to `KnowledgeBase`. Build one `KnowledgeBase` and pass it to every parser, engine, and document that should interoperate, for example `Parser::builder().knowledge_base(kb.clone()).build()` and `TransformEngine::builder().knowledge_base(kb)`. `ParserBuilder::build()` now returns `Parser`, and `ParserBuildError` and `Error::ParserBuild` are removed. `TransformEngine::transform` accepts any document bound to the same instance and returns `Error::KnowledgeBaseMismatch` otherwise, replacing `Error::ForeignDocument`. In Python, pass `texform.KnowledgeBase(packages, items=...)` to `Parser(kb)` and `TransformEngine(profile, kb)`; in JavaScript, pass `{ knowledgeBase }`. The command-line tool and the `serve` protocol are unchanged. - **Breaking:** Construction, edits, and syntax imports keep every complete document conformant to its knowledge base. A violation returns `ConformanceError` with a rooted path and a stable rule code (`TexformConformanceError` in JavaScript) and leaves the document unchanged. Replace `ArgValue` with `Arg`, and binding argument dictionaries with `Node | str | bool | None | Paired` values (`{ value, open, close }` pairs in JavaScript). `create_group` and `create_environment` take their children and body directly, binding delimiters are strings such as `"("` and `"\\langle"`, and environments are renamed with `set_env_name` instead of `set_command_name`. `Document::from_syntax` rejects non-conformant complete trees, with `FromSyntaxError::Conformance` replacing the prime-specific variants, and `KnowledgeBaseBuilder` rejects item names the lexer cannot produce with `KnowledgeBaseBuildError::InvalidName`. ### Added - Every node kind has a constructor, including scripted nodes, infix commands, primes, inline math, and delimited groups. Commands and environments are built from their knowledge-base records, so optional, star, and paired slots can be filled, and a constructor called with a name of another command kind names the one to use. Arguments are given as nodes, source text parsed in the slot's mode, star Booleans, or explicit absent and paired values; `in_mode` and `parse_fragment` construct in an explicit context mode. - Deterministic edits: `set_subscript` and `set_superscript` add, change, or clear scripts, collapsing the wrapper when the last script is cleared; `set_delimiters`, `set_arg_delimiters`, and `set_prime_count` change delimiters and prime counts. `clone_node` copies a subtree within a document, and `import_node` copies one from another document, validating it against the destination knowledge base. Failed edits leave the document unchanged. - `NodeRef::path` returns a node's rooted path and `Document::node_at` resolves it; `NodeRef::slot` and `NodeRef::is_known` report the parent slot and knowledge status. Python and JavaScript expose the same edits and reads, JavaScript adds `argKind(index)`, and argument reads include their slot form. - `Document::to_columnar` exports a document as a preorder node table and an argument table for Arrow and DataFrame tools (`to_columnar()` in Python, `toColumnar()` in JavaScript); Python tables pass directly to `pyarrow.table`. - `KnowledgeBase` instances are cheap to clone and compare by identity (`KnowledgeBase::ptr_eq`); `KnowledgeBase::default()` returns one process-wide instance, and `Document::with_knowledge_base` and `Document::from_syntax_with` bind an explicit one. Python and JavaScript expose `KnowledgeBase` with identity equality and hashing, document copies (`copy()` in Python, `clone()` in JavaScript), node identity, and `document()` on nodes. ### Fixed - In alignments, a script directly after a column separator (`&^2`, `&_1`, `&'`) now starts the next cell with an empty base instead of using `&` as its base. Escaped `\&` still takes scripts. - An unbraced argument of a text-mode command takes a single token, as in TeX and MathJax: `\mbox TeV` parses as `\mbox{T}eV` instead of `\mbox{TeV}`. `\text{$x$ }` and `\textbf{ \emph k }` now parse. - Alignment separators (`&`, `\\`, `\newline`, `\cr`) end declarative scopes in environment bodies and math arguments, so `\begin{array}{cc} \rm a & b \\ c & d \end{array}` lowers to `\mathrm{a} & b \\ c & d` instead of the invalid `\mathrm{a & b \\ c & d}`, and a `\displaystyle` repeated in a later cell is kept. ## 0.7.0 · 26 Sep 2026 This release makes normalized output reparse to the same tree, so normalizing it again no longer changes it, and stops math-mode `\&` from turning into an alignment tab. Both fixes change the tree shape; see **Changed** for migration notes. ### Changed - **Breaking:** A bare `&` now parses as a new `AlignmentTab` node in `SyntaxNode`, `Node`, and `NodeKind`, and `Char('&')` always means a literal ampersand. Code that matched or built `Char('&')` for an alignment tab should use `AlignmentTab`, and exhaustive matches need the new variant. `Document::create_alignment_tab` stages one (`create_alignment_tab` in Python, `createAlignmentTab` in JavaScript); node kinds report `"AlignmentTab"` in Python and `"alignmentTab"` in JavaScript. - **Breaking:** A command argument whose content is a single brace group now parses to an implicit container around that group, so a brace group directly in any slot owns that slot's braces. FlattenGroups always unwraps redundant user braces in argument slots, and the `command_argument` guard and its report field are removed from Rust, Python, and JavaScript. - The default packages are now `base`, `ams`, `braket`, `textmacros`, `bboldx`, and `boldsymbol`; `physics` must be loaded explicitly. ### Fixed - Normalizing the output of `normalize` again no longer changes it. Argument braces around a single brace group are kept (`\overline{{{\Psi}}}`, `\mathrm{{}}`), braces that protect a closing delimiter inside an optional argument are restored on output (`\xrightarrow[{]}]{x}` no longer becomes `\xrightarrow [ ] ] { x }`), and a lone `\` at a line end or input end is a control space, as in TeX. - Optional, `Delimited`, and `Paired` arguments track brace depth, so protected content such as `\qty[{]}]` parses. - In text mode, whitespace after an argument-free control word is no longer content: `\text{\bf\large x}` no longer normalizes to `\textbf{ x}`. - Math-mode `\&`, such as `a \& b` or `\mathrm{\&}`, is serialized as `\&` instead of a bare `&` that became a column separator inside `matrix` or `align`. All profiles were affected, including plain parse and serialize. ## 0.6.0 · 25 Sep 2026 This release adds the `texform` command-line tool, gives prime marks a consistent meaning as symbols with explicit superscript binding, and makes normalization stable across transform phases. It also fixes spacing drift after text-mode control symbols and declarations leaking out of local groups, and roughly halves transform time. The prime change is breaking; see **Changed** for migration notes. ### Added - A new `texform-cli` crate installs a `texform` binary (`cargo install texform-cli`). `normalize`, `parse`, and `tokenize` read one formula from an argument or stdin, or one per line with `--lines`, and print human-readable text or one JSON object per formula; `info`, `packages`, and `argspec validate` query the knowledge base and argument specifications with the same shapes as the bindings. `texform serve` speaks JSON-RPC 2.0 over stdio (normalizer protocol v1) so external tools can call a specific TeXForm build across a process boundary, and `texform --version` reports the source commit and whether it was dirty. The CLI loads the library's default packages; `braket` is opt-in. ### Changed - **Breaking:** `Prime` is now a math symbol, and only `Scripted` expresses superscript binding. `x\prime` stays `x \prime` instead of becoming `x'`, `x^2\prime` no longer produces an unparsable double superscript, `f'^2` serializes as one superscript instead of a nested empty-base one, and `f^{'}` no longer collapses to `f'`. A pure prime superscript still serializes as `f'`. A leading quote, including inside `f^{'}` and `A^{'\alpha}`, parses as an empty-base `Scripted`, and whitespace-separated quotes such as `f' ' '` form one superscript as in MathJax. `x^'` and `x_'` are now parse errors. Snapshots that stored a bare `Prime` for a quote superscript should be reparsed from source or rebuilt with an empty-base `Scripted`. - Transforms are faster: rewrite trigger indexes are built once per `Plan`, and phases skip work they have already done. On a 10,000-formula sample, transform time falls from 33–44 µs to 14–18 µs per formula across the four profiles. ### Fixed - Normalization no longer depends on how many times it is applied. Later phases could expose attribute merges or rewrite matches after earlier phases had finished; the phases now run to a shared fixed point, and a transform that fails to converge within eight rounds returns a `TransformError` instead of partially normalized output. As a result, a prefix command's single-node argument is no longer wrapped in an implicit group in the transformed tree; serialized output is unchanged. - Text-mode control symbols such as `\ ` and `\,` no longer gain a separator space on every serialization (`\mbox{mod\ 1}` grew by one space per round). A separator is emitted only where it keeps a control word from absorbing the following letter. - Declaratives without a prefix form keep their local scope: `{\scriptstyle x} y` no longer becomes `\scriptstyle x y`, `{\oldstyle 1}2` no longer becomes `\oldstyle 12`, and `\mathbf{\scriptstyle x}y` no longer emits the style twice. ## 0.5.0 · 22 Sep 2026 This release reshapes the configuration and reporting surface across Rust, Python, and JavaScript so the three APIs share one configuration tree and one set of override semantics. Transform reports become opt-in, `TransformConfig` is organized by phase, serialization options are flattened, and binding exception categories are corrected. It also hardens parser recovery and diagnostics, fixes serialization of bare delimiter arguments, and adds two corpus rules. Most changes are breaking; see **Changed** for migration notes. ### Added - `normalize_with_report` / `transform_with_report` on the Rust facade and the Python binding (`normalizeWithReport` / `transformWithReport` in JavaScript) for callers that need the diagnostic report. Report types now live in `texform::diagnostics`. - Effective defaults are exposed on every binding: `default_parse_config()` / `default_transform_config()` in Python and `defaultParseConfig()` / `defaultTransformConfig()` in JavaScript return the engine's profile defaults. Rust adds `TransformEngine::default_normalize_config()`. - `RewriteConfig`, plus the serialization option enums (`CommandSpacing`, `MathGroupInnerSpacing`, `AdjacentCharSpacing`, `ScriptSpacing`, `ScriptOrder`, `InfixGrouping`, `EnvironmentNameSpacing`), are re-exported from the Rust facade. - Corpus rules `leftroot-drop` and `uproot-drop` drop the AMS `\leftroot` / `\uproot` index-position hints. ### Changed - **Breaking:** transform reports are opt-in. `normalize` / `normalize_with` now return the canonical string and `transform` / `transform_with` return nothing, in Rust, Python, and JavaScript, so plain calls no longer pay for report collection or host-object conversion. `NormalizeResult` and the `FinalizeAstStep*` types are removed. The report itself is restructured (`rewrite.{iterations, rules}`, flattened FinalizeAst counters, `flatten_groups.guard_hits` with short guard names) and remains a diagnostic rather than a stability promise. - **Breaking:** `TransformConfig` is organized by phase. The flat `rewrite_enabled`, `lower_attributes_enabled`, and `max_iterations` fields become `rewrite: RewriteConfig { enabled, max_iterations }` and `lower_attributes: LowerAttributesConfig`, alongside the existing `finalize_ast` and `flatten_groups`. Profile defaults are unchanged. - **Breaking:** `FlattenGroupsConfig` exposes only `enabled` and `preserve_rendered_spacing` (`preserveRenderedSpacing` in JavaScript). The eleven `preserve_*` guard fields are removed and old keys are rejected. - **Breaking:** serialization options are a flat set of seven fields; the nested `math` / `syntax` shape is removed. Python takes snake_case keyword arguments (`doc.to_latex(script_order="sup_first")`, `serialize(node, **options)`), and JavaScript takes a flat camelCase object. - **Breaking:** configuration inputs are unified across bindings. Python reserves `config=` for a complete config object (`TransformConfig`, `ParseConfig`); partial settings are passed as keyword overrides that keep unspecified defaults, and a dict passed as `config` is rejected. JavaScript accepts the same nested override tree in `normalize` and `transform`, with omitted fields keeping the profile default. Unknown keys and wrongly typed values raise `ConfigError` / `TexformConfigError` with the field path. The unused WASM config classes and `parseWith` are removed. - **Breaking:** binding exception categories are corrected. Python cross-document edits raise `EditError` instead of `ParseError`. Transforming a document that has parse errors raises `TransformError` in Python and `TexformTransformError` (`kind: "transform"`) in JavaScript instead of the `internal` kind. The undocumented `Complete` type is no longer importable from the npm typings. ### Fixed - Parser recovery under `reject_unknown` without `abort_on_error` consumes only the unknown command, so `\mycmd{x} + y` yields a partial document instead of failing the whole parse. A stray `}` is no longer reported as an environment-name mismatch, and `^` / `_` with missing content report `Missing superscript content` / `Missing subscript content`. - Diagnostics keep their structured kind and precise source span through nested content parsing instead of highlighting the outer command or closing brace, and internal kind labels no longer leak into messages. - Literal carriage returns are treated as whitespace, so Windows (CRLF) and CR-only inputs parse instead of panicking. - Mandatory delimiter arguments are serialized without braces (`\middle \vert`, `\big \langle`, `\genfrac . . {} {}`), so the output renders in MathJax again. - JavaScript: `normalize` with a `flattenGroups` override now layers it on the engine profile's defaults instead of the strict baseline. - JavaScript: internal DTO serialization failures are thrown as `TexformError` (kind `internal`) instead of being returned as successful values. - Python: invalid per-call config objects raise `ConfigError` consistently across `parse`, `normalize`, and `transform`. ## 0.4.0 · 26 Jul 2026 This release aligns `TransformEngine` parsing with the standalone `Parser` by defaulting to lenient parsing, and fixes prime superscripts inside environment bodies. ### Changed - **Breaking:** `TransformEngine` now parses with `ParseConfig::LENIENT` by default (previously `ParseConfig::STRICT`), matching the standalone `Parser`; this applies to Rust, Python, and JavaScript. Unknown commands are preserved instead of rejected, so `normalize(r"\unknowncmd")` now succeeds, and `engine.parse` returns a recovery document with error nodes for malformed input. `normalize` still requires a complete tree and raises a parse error, carrying the diagnostics and the partial document, when the input cannot produce one. To restore the previous behavior, pass `ParseConfig::STRICT` as the engine's default parse config, or set `reject_unknown` and `abort_on_error` to `true` (`rejectUnknown` / `abortOnError` in JavaScript). ### Fixed - A lone prime inside an environment body, such as an `array` cell, no longer serializes as an explicit superscript group that renders as a double superscript. FlattenGroups now unwraps lone `Prime` nodes in superscript slots while still preserving other environment-body groups. ## 0.3.0 · 21 Jul 2026 This release adds a tokenized serialization channel, extends AST canonicalization with text-sequence normalization, and substantially expands the rewrite rule set — spacing, fraction and delimiter styling, negated-relation remaps, named-function operators, and more. It also renames the rule-level taxonomy on the public API and prunes several rules whose render fidelity did not hold up under corpus review. ### Added - Tokenized serialization. `Document::to_tokenized_latex()` (and `to_tokenized_latex_with`) returns the canonical LaTeX string alongside ordered, typed output tokens — each classified as `ControlSequence`, `Character`, `Delimiter`, `Text`, `Raw`, or `Error`, and carrying its math/text mode and a non-overlapping UTF-8 byte span into that string. Tokens are recorded during the existing serializer traversal rather than by re-lexing the output, and the feature is exposed across the Rust, Python, and JavaScript APIs while leaving text-only serialization's allocation behavior unchanged. - Text-sequence normalization in `FinalizeAst`. The profile-neutral canonicalization phase now merges adjacent text-mode siblings, collapses ordinary lexer whitespace runs to a single space without trimming edges, and cleans empty text. The pass re-runs after `FlattenGroups` so newly adjacent text and prime nodes are canonicalized, and its work is reported through the `normalize_text_sequences` step counter on the transform report. - A large batch of rewrite rules across several normalization groups: spacing drops (`\enspace`, `\quad`, `\qquad`) and merges (adjacent `\enspace` pairs, small-spacer runs); fraction styling (`\dfrac`/`\tfrac`/`\cfrac` → `\frac`, `\dbinom`/`\tbinom` → `\binom`); limit placement (`\limits`/`\nolimits` drops on audited operators); fixed delimiter sizing (`\big`–`\Bigg` size drops); negated-relation remaps (`\not=` → `\neq`, `\not\exists` → `\nexists`, `\not\in` → `\notin`, `\not\rightarrow` → `\nrightarrow`); accent sizing (`\widehat` → `\hat`, `\widetilde` → `\tilde`, the first `Equiv`-level builtins); math-mode control-space to active space; duplicate `mathrel`-wrapper drops; and named-function rewrites for plain operator names, with extended `\operatorname` canonicalization. Shared delimiter-rewrite helpers back the delimiter-oriented rules. ### Changed - **Breaking:** the rule-level taxonomy is renamed from *normalization level* to *rule level* across the public API, the transform engine, the generated registry, rule metadata, and documentation. The facade now re-exports `RuleLevelSet` in place of `NormalizationLevelSet`, and the internal `NormalizationLevel` enum becomes `RuleLevel`; downstream code referring to the old names must update. Active rules are reclassified under `Authoring`, `Faithful`, `Corpus`, and `Equiv` while preserving current profile behavior. - `\limits`/`\nolimits` drops move from `Equiv` to `Corpus`, so they now participate in corpus normalization. ### Fixed - Removed rewrite rules whose worst-case render fidelity did not hold across supported renderers: the `displaylines-to-gather-env` rule (its numbered `gather` output cannot preserve reading fidelity), the `repeat-spacer-collapse` rule, and the physics quick-`\quad` expansions (`qcomma-expand`, `qqtext-expand`). These forms are now preserved instead of rewritten. ## 0.2.1 · 16 Jul 2026 This release expands the normalization rule set with spacing canonicalization and a batch of alias and macro-expansion rules, so more spacing markup, legacy shorthand, and convenience commands collapse to canonical, universally renderable forms. ### Added - Spacing normalization rules. Adjacent `\quad` pairs merge into `\qquad` and repeated small spacers collapse; MathJax spacing aliases — `\thinspace`, `\negthinspace`, `\hskip`, `\mkern`, `\mskip`, `\hfil`, `\hfilll`, `\space`, `\nobreakspace`, and `\gt` — map to their canonical spellings while preserving scalar dimensions and documented fidelity boundaries; and pure layout hints (`\mathstrut`, `\strut`, and line-break hints) are dropped. - Alias and macro-expansion rules that fold convenience and legacy forms into universal output: `\ast` canonicalizes to the pixel-identical literal `*`; `\dots` resolves to `\ldots` or `\cdots` from the surrounding atom class, leaving unclassifiable and boundary cases untouched; `\impliedby` expands to the source-equivalent spaced `\Longleftarrow`; the `Vmatrix` environment expands into a matrix wrapped in explicit double-bar fences; and the `\bigl`/`\bigr`/`\bigm` class variants — with their `\Big`, `\bigg`, and `\Bigg` counterparts — collapse to the plain `\big`-family delimiters. ### Fixed - Rule metadata is aligned with the published rule proposals, keeping each rule's declared normalization level and fidelity in sync with its documented contract. - Rewrite rules that synthesize new commands — the implies, derivative, eval, and multi-integral expansions — now declare every character they emit, drawing on builtin character records so the engine's eliminated-form contract sees their full output. ## 0.2.0 · 3 Jul 2026 This release adds in-place document normalization to the Python and JavaScript bindings, teaches the parser and serializer to preserve whitespace and spacing faithfully, and makes parsing and transforming large formulas dramatically faster. ### Added - `TransformEngine.transform` on the Python and JavaScript bindings, for normalizing a live `Document` in place. Every parsed document is stamped with a parse-context id, and a document produced by a different parser (or by `Document.from_syntax`) is rejected with a foreign-document error, so a document is only transformed by the engine that produced it. - `:O` operator-name argspec content, so arguments to `\operatorname` and `\DeclareMathOperator` are modeled as math content and serialized compactly without special-casing command names. - Typed serialization options for the Python bindings: `serialize()` and `Document.to_latex()` now accept a documented `SerializeOptions` TypedDict instead of an untyped `dict`, mirroring the TypeScript option interfaces. ### Changed - **Breaking:** the public `Error` enum is now `#[non_exhaustive]`. Exhaustive `match` arms over `texform::Error` in downstream Rust must add a wildcard arm; in return, future error variants can be introduced without another breaking change. - Parsing and transforming large formulas is dramatically faster. Release builds no longer run debug-only structural invariant sweeps, which were quadratic on wide formulas (a 60k-character formula now parses ~14× faster and transforms ~45× faster); source spans are carried as a positional tree (10–25% faster parsing); and rewrite rules are indexed by trigger name (31–48% faster transforms). Normalized output is byte-identical. ### Fixed - Edge whitespace in text arguments is preserved: `\text{ or }` and `\textbf{ a }` no longer drop their leading and trailing spaces on parse, serialize, or transform. - Adjacent math digits stay compact: multi-digit numbers such as `1093^2` no longer serialize as `1 0 9 3 ^ { 2 }` under the default spacing option, while letters and symbols still honor it. - Tight argspec slot spacing is preserved, so no-leading-space slots — including linebreak dimensions and custom tight optional slots — stay tight through parse/serialize round trips. - Whitespace is kept outside attribute wrappers during transforms. - Inline math inside text-mode control sequences, and whitespace-only text arguments, are now accepted. - The recovery parser is hardened against an unnecessary unwrap. - The license file is included in the Python sdist. ## 0.1.0 · 12 Jun 2026 Initial public release of TeXForm — a LaTeX formula parser, editable document model, and normalization engine, available in Rust, Python, and JavaScript from a single Rust core. ### Added - Knowledge-driven parser backed by 530+ command and environment specifications across the `base`, `ams`, `physics`, `braket`, `bboldx`, `boldsymbol`, and `textmacros` packages, with strict and lenient modes that preserve unknown commands and unparseable fragments as explicit nodes instead of failing the parse. - Editable `Document` tree with validated, fallible edits and canonical LaTeX serialization that guarantees text idempotency over parse/serialize cycles. - Profile-based transform engine with four normalization profiles — `Authoring`, `Faithful`, `Corpus`, and `Equiv` — covering author-facing cleanup, render-faithful expansion, corpus preparation, and formula-equivalence comparison. - `validate_argspec` for checking xparse-style argument specifications. - Python (PyPI `texform`, Python ≥ 3.10) and JavaScript/TypeScript (npm `texform`, WebAssembly) bindings exposing the same parser, document, and transform engine from the shared Rust core. --- # Stability and versioning What the stability guarantee covers, lockstep versioning across Rust, Python, and JavaScript, and what 0.x means. https://texform.dev/docs/stability-and-versioning ## What the guarantee covers [#what-the-guarantee-covers] The `texform` crate is the only Rust crate with a public stability guarantee. The other crates in the workspace (`texform-core`, `texform-transform`, `texform-knowledge`, and the rest) are internal: their APIs may change without notice, and external integration must go through the `texform` facade. The Python package (`texform` on PyPI) and the JavaScript package (`texform` on npm) expose that same parser, document model, transform engine, and serializer, and carry the same guarantee: public names, behavior, and data shapes documented in the [API reference](https://texform.dev/api) are covered. ## Lockstep versioning [#lockstep-versioning] One version number covers all three release channels. A given version of the Rust crate, the Python wheel, and the npm package are built from the same source and behave identically. There is no compatibility matrix to consult: matching versions match behavior. ## Semantics during 0.x [#semantics-during-0x] TeXForm follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html). While the major version is 0: * **Minor releases (0.x → 0.y)** may contain breaking changes to the public API or to normalization output. Breaking changes are always marked explicitly in the changelog. * **Patch releases (0.x.y → 0.x.z)** contain fixes and additions only. Normalization output is part of the observable behavior: a change that alters what a profile produces for existing input is treated as a behavior change and recorded in the changelog, even when no API signature changed. ## Where changes are recorded [#where-changes-are-recorded] Every release documents user-visible changes in the [changelog](https://texform.dev/docs/changelog), grouped as Added, Changed, and Fixed. Internal refactors that do not change observable behavior are not listed. --- # Performance and correctness How TeXForm checks parser and transform correctness, reference throughput and startup measurements for each binding, and how to measure on your own input. https://texform.dev/docs/performance-and-correctness TeXForm checks correctness with corpus regression, transform contracts, and renderer validation. These checks cover different properties; none alone establishes that a normalization is suitable for every input. ## How correctness is checked [#how-correctness-is-checked] * **Corpus regression.** Every significant parser change is run against tracked real-world datasets, and failure rates are compared before and after. * **The eliminated-forms contract.** Each rewrite rule declares which source forms it promises to remove, and the engine verifies after the full pipeline that those forms are gone. The check is re-run across the same corpora. See [Rules and the transform contract](https://texform.dev/docs/rule-contract). * **Text idempotency.** Serialization guarantees that a second parse-and-serialize round trip is a fixed point. See [Serialization](https://texform.dev/docs/serialization). * **Renderer-validated knowledge base.** Command signatures are validated by rendering generated test cases against real renderers rather than transcribed from documentation. See [The knowledge base](https://texform.dev/docs/knowledge-base). ## Parse failure rates [#parse-failure-rates] Strict parsing rejects unknown commands and stops at syntax errors. Lenient parsing preserves unknown commands and recovers where possible. A strict failure may indicate an unsupported command rather than malformed input; a recovered document may still contain errors. Measure both on input representative of your application, and inspect diagnostics before transforming a document. See [Handle parse errors](https://texform.dev/docs/handle-parse-errors). ## Eliminated-form contract [#eliminated-form-contract] The contract checker runs a profile and checks that forms its rules promise to eliminate are absent from the output. A successful check establishes that property for the checked input; it does not establish mathematical equivalence or renderer compatibility. See [Rules and the transform contract](https://texform.dev/docs/rule-contract) for rule metadata and the scope of the guarantee. ## Performance [#performance] ### Reference measurements [#reference-measurements] The tables below show what to expect from each binding on a public dataset. They come from one machine, so read them for orders of magnitude and for the differences between bindings, not as a promise for your hardware. Throughput, in formulas per second, for parsing alone and for normalizing with each profile: Time per formula: Parsing accounts for most of the work, so the four profiles normalize at similar rates. The Python binding calls the same native core as Rust and stays close to it; the WebAssembly build is slower and its first calls also pay for loading the module and warming up the JavaScript engine. One-time costs in a fresh process, before the first formula is processed: ### Measuring your own workload [#measuring-your-own-workload] Measure throughput on your own hardware and representative input, using the profile and package selection your application needs. Separate engine construction from repeated normalization when reporting timings. Two things do have a large and predictable effect on throughput. **Reuse the engine.** Constructing a `Parser` or `TransformEngine` loads and indexes the knowledge base and builds the rule plan for the profile. That cost belongs once per process, not once per formula: build the engine outside your loop and call `normalize` inside it. [Normalize a corpus](https://texform.dev/docs/normalize-a-corpus) shows the shape. **Build in release mode if you build from source.** Debug builds run structural invariant sweeps that release builds omit, and those sweeps are quadratic on wide formulas. The published wheels and the npm package are release builds already; this matters only if you compile the crate yourself. The bindings are synchronous, and there is no batch or parallel API to reach for. Parallelism, when you need it, is your own — partition the input and give each worker its own engine. ## Next steps [#next-steps] - [Normalize a corpus](https://texform.dev/docs/normalize-a-corpus) — The reused-engine loop, with error handling and rule statistics. - [Rules and the transform contract](https://texform.dev/docs/rule-contract) — What a rule promises to eliminate and how the engine verifies it. - [The knowledge base](https://texform.dev/docs/knowledge-base) — Where entries come from and how their signatures are validated. --- # API Reference Use TeXForm from Rust, Python, or JavaScript. Three bindings, one Rust core. https://texform.dev/api The same Rust core sits behind all three bindings, so the parser, document model, transform engine, and serializer behave identically across languages. The Python and JavaScript references below are generated from the binding comments; the Rust API lives on docs.rs. * [Python](https://texform.dev/api/python) — the `texform` package: Parser, Document, Node, TransformEngine, and serialization. * [JavaScript](https://texform.dev/api/js) — the `texform` npm package for Node and the browser. * [Rust](https://docs.rs/texform) — the core crate shared by all bindings. ## Naming across bindings [#naming-across-bindings] Only the surface naming differs. Python uses snake_case for methods, arguments, and option keys (`to_latex`, `has_errors`, `reject_unknown`); JavaScript uses the camelCase equivalents (`toLatex`, `hasErrors`, `rejectUnknown`). Enum *values* stay snake_case in both (for example, `"spaced"` and `"compact"`). Structured data crosses the boundary as plain dicts and objects rather than wrapper classes, while `Document` and `Node` are live handles into the Rust tree in both bindings. Start with the overview page for your language: it states the conventions, the error hierarchy, and a map of every exported symbol. To install a binding first, see [Installation](https://texform.dev/docs/installation). For how the bindings wrap one core without copying trees across the language boundary, see [Architecture](https://texform.dev/docs/architecture). --- # Python API Python conventions, the exception hierarchy, and a map of the texform module. https://texform.dev/api/python The `texform` Python package exposes the parser, document model, transform engine, and serializer from one Rust core. Install it with `pip install texform`; see [Installation](/docs/installation) for supported Python versions and platforms. Methods and arguments use snake_case; the JavaScript binding uses camelCase. Both bindings share the parsing and transform core, but configuration interfaces and some exception types differ. Structured data crosses the boundary as plain dicts, never custom wrapper classes: parse results, transform reports, diagnostics, and argspec results are all dicts, and option inputs are dicts too. The bundled type stubs (`texform/__init__.pyi`) describe those shapes as `TypedDict` types, so editors and type checkers see the exact keys. `Document` and `Node` are live handles into a Rust tree; a `SyntaxNode` is a plain dict snapshot. Configuration is two-layered. A per-call `config=` argument must be a complete configuration class — `ParseConfig` or the phase-nested `TransformConfig` — and a dict there raises `ConfigError`. Partial adjustments go through keyword overlays instead (`parse(src, reject_unknown=True)`, `normalize(src, rewrite={"enabled": False})`), where nested values are plain dicts, not phase-config instances. The parse baseline is lenient. The structured library exceptions derive from `TexformError`, so a single `except texform.TexformError` catches them: `ParseError` (an operation needed a complete tree but parsing produced none), `EditError` (an invalid edit), `ConfigError` (invalid construction input), and `TransformError` (a transform-engine failure). `Parser.parse` returns malformed-input diagnostics and a `document` that may be `None`; invalid configuration still raises `ConfigError`. ## Modules ### [Parser](https://texform.dev/api/python/parser.md) - `Parser` (class): Turn LaTeX source into a parse result. - `ParseConfig` (class): Complete parse configuration. The only accepted `config=` type for parse. - `ParseOverrides` (typedDict): Keyword overlays for `parse`. Omitted keys keep the baseline. - `ParseResult` (typedDict): The result of `Parser.parse`: an optional document plus diagnostics. - `ParseDiagnostic` (typedDict): A single parse diagnostic emitted alongside a parse result. - `ParseDiagnosticKind` (typeAlias): Category of a parse diagnostic, such as `"unknown-command"` or `"argument-validation"`. - `Span` (typeAlias): Half-open UTF-8 byte range `{"start": int, "end": int}` into the source. - `SyntaxNode` (typeAlias): Lossless parse snapshot as a plain dict. The tagged-union shape is documented on the TypeScript `SyntaxNode` type; Python keeps this alias unexpanded. - `ContextItem` (typeAlias): A knowledge-injection item: a command, environment, or delimiter-control dict. - `CommandItem` (typedDict): A custom command injected into parser knowledge. - `EnvironmentItem` (typedDict): A custom environment injected into parser knowledge. - `DelimiterItem` (typedDict): A custom delimiter control injected into parser knowledge. ### [KnowledgeBase](https://texform.dev/api/python/knowledge-base.md) - `KnowledgeBase` (class): Immutable knowledge shared by parsers, engines, and documents. - `CommandItem` (typedDict): A custom command injected into parser knowledge. - `EnvironmentItem` (typedDict): A custom environment injected into parser knowledge. - `DelimiterItem` (typedDict): A custom delimiter control injected into parser knowledge. ### [Document](https://texform.dev/api/python/document.md) - `Document` (class): The editable document tree — the working format you read, edit, and output. - `NodeSpanEntry` (typedDict): One entry mapping a node identifier to its source byte span. - `TokenizedLatex` (typedDict): Serialized LaTeX and tokens produced by the same serializer traversal. - `SerializationToken` (typedDict): One serialization fragment with UTF-8 byte offsets. - `SerializationTokenKind` (typeAlias): Semantic category of one serialization fragment. - `NodeTable` (typedDict): Preorder node columns. Row indices are local to this snapshot. - `ArgumentTable` (typedDict): All argument slots, ordered by owner row and slot index. - `ColumnarTree` (typedDict): Tree columns for bulk structural analysis with Arrow or DataFrame tools. ### [Node](https://texform.dev/api/python/node.md) - `Node` (class): A read-only handle into a `Document` for navigation and reading. - `NodeKind` (typeAlias): Node kind as reported by `Node.kind()`. Python uses PascalCase (`"Root"`, `"Command"`, `"ActiveSpace"`). - `RuntimeContentMode` (typeAlias): Runtime content mode reported by the tree API: `"math"` or `"text"`. - `ErrorParts` (typedDict): The decomposed content of an `Error` placeholder node. - `GroupKindExplicit` (typedDict): A group written with explicit braces `{ ... }`. - `GroupKindImplicit` (typedDict): A group with no surrounding braces, inferred from structure. - `GroupKindDelimited` (typedDict): A group bounded by a delimiter pair, such as `\left( ... \right)`. - `GroupKindInlineMath` (typedDict): A group introduced by inline math shift, such as `$ ... $` in text mode. - `GroupKindRef` (typeAlias): The kind of a `Group` node: explicit, implicit, delimited, or inline-math. - `DelimiterValue` (typeAlias): A delimiter string: a character, a backslash-prefixed control, or a dot. - `MathArg` (typedDict): A command argument carrying math-mode content. - `TextArg` (typedDict): A command argument carrying text-mode content. - `DelimiterArg` (typedDict): A command argument that is a single delimiter token. - `CSNameArg` (typedDict): A command argument that is a control-sequence name. - `DimensionArg` (typedDict): A command argument that is a TeX dimension, such as `2pt`. - `IntegerArg` (typedDict): A command argument that is an integer literal. - `KeyValArg` (typedDict): A command argument that is a `key=value` list. - `ColumnArg` (typedDict): A command argument that is a tabular column specification. - `BooleanArg` (typedDict): A command argument that is a star flag, modeled as a boolean. - `ArgRef` (typeAlias): A read command argument: one of the discriminated argument-value dicts. - `ArgFormKind` (typeAlias) - `NodeSlotKind` (typeAlias) - `ArgForm` (typedDict) - `OperatorNameArg` (typedDict) - `Paired` (class): Immutable content and delimiter pair for an argspec Paired slot. - `Arg` (typeAlias): Slot input: source written inside the slot, a detached content node, a star flag, absence, or an explicit `Paired` delimiter pair. - `NodeSlot` (typedDict) ### [TransformEngine](https://texform.dev/api/python/transform-engine.md) - `TransformEngine` (class): Normalize a formula into the canonical form selected by a profile. - `TransformConfig` (class): Complete transform configuration, nested by pipeline phase. - `RewriteConfig` (class): Complete Rewrite phase configuration. - `LowerAttributesConfig` (class): Complete LowerAttributes phase configuration. - `FinalizeAstConfig` (class): Complete FinalizeAst phase configuration. - `FlattenGroupsConfig` (class): Complete FlattenGroups phase configuration. - `NormalizeOverrides` (class): Flat parse+transform overlays for `normalize`. - `TransformOverrides` (typedDict): Keyword overlays for `transform`. - `RewriteOverrides` (typedDict): Keyword overlays for the Rewrite phase. - `LowerAttributesOverrides` (typedDict): Keyword overlays for the LowerAttributes phase. - `FinalizeAstOverrides` (typedDict): Keyword overlays for the FinalizeAst phase. - `FlattenGroupsOverrides` (typedDict): Keyword overlays for the FlattenGroups phase. - `NormalizeReportResult` (typedDict): Text plus the diagnostic report from one explicit normalize report call. - `TransformReport` (typedDict): Diagnostic report for one explicit report call. - `TransformWarning` (typedDict): An unknown input name whose arguments or scope cannot be guaranteed. - `LowerAttributesReport` (typedDict): The LowerAttributes phase report. - `AttributeStat` (typedDict): Activity for one `(attr, value)` pair in LowerAttributes. - `AttributeFormCounts` (typedDict): Counts of one attribute form, split by carrier. - `RewriteReport` (typedDict): The Rewrite phase report. - `RewriteRuleStat` (typedDict): One rewrite rule's outcome for a single call. - `FinalizeAstReport` (typedDict): The FinalizeAst phase report. - `FlattenGroupsReport` (typedDict): The FlattenGroups phase report. - `FlattenGroupsActionCounts` (typedDict): Flatten actions that changed the tree. - `FlattenGroupsGuardCounts` (typedDict): Preserve-guard hits for one FlattenGroups run. - `TransformProfile` (typeAlias): Normalization profile name: `"authoring"`, `"faithful"`, `"corpus"`, or `"equiv"`. ### [serialize](https://texform.dev/api/python/serialize.md) - `serialize` (function): Render a `SyntaxNode` dict to LaTeX text. - `SerializeOptions` (typedDict): Options for `Document.to_latex`, `Document.to_tokenized_latex`, and `serialize`. - `Style` (typeAlias): Spaced output is byte-stable; compact preserves structure and text idempotency. ### [validate_argspec](https://texform.dev/api/python/validate-argspec.md) - `validate_argspec` (function): Validate an xparse-style argspec string and report its parsed slots. - `ValidateArgspecResult` (typedDict): The outcome of validating an argspec string with `validate_argspec`. - `ParsedArgSpecSlot` (typedDict): One parsed argument slot of an xparse-style specification. - `ArgSpecKindType` (typeAlias): Value kind of one parsed argspec slot (content, delimiter, star, ...). - `ArgSpecFormType` (typeAlias): Surface form of one parsed argspec slot (mandatory, optional group, paired, ...). ### [Utilities](https://texform.dev/api/python/utilities.md) - `list_packages` (function): List the built-in knowledge packages with their record counts. - `count_targets` (function): Count command, environment, and character targets in a LaTeX formula. - `PackageInfo` (typedDict): Summary of one built-in knowledge package. - `TexformError` (exception): Base class for every exception the library raises. - `ParseError` (exception): Raised when source could not be parsed into a complete tree. - `EditError` (exception): Raised by a `Document` editing method on an invalid edit. - `ConfigError` (exception): Raised on invalid construction or per-call configuration input. - `TransformError` (exception): Raised on a transform-engine failure. - `ConformanceError` (exception): Raised when a construction, edit, or syntax import violates the knowledge base or the document's structure. - `list_rules` (function): List the built-in rewrite rules, sorted by key. - `RuleInfo` (typedDict): Summary of one built-in rewrite rule. - `RuleFidelity` (typeAlias): A rewrite rule's worst-case equivalence guarantee, least to most faithful: `"math"` (meaning preserved, rendering may change), `"reading"` (notation content, reading order, and structural roles preserved), or `"render"` (rendering equivalent under the reference renderer). - `RuleLevel` (typeAlias): The first profile that accepts a rewrite rule's output as a suitable product. --- # Parser Construct a Parser, parse LaTeX, configure strictness, and inspect the knowledge base. https://texform.dev/api/python/parser ## `Parser` Kind: class ```python Parser(knowledge_base: KnowledgeBase | None = None, default_parse_config: ParseConfig | None = None) -> None ``` Turn LaTeX source into a parse result. A ``Parser`` consults the knowledge base for command and environment signatures to build structured parse trees. It never raises on malformed input and never fabricates a placeholder tree; instead it reports diagnostics and, in lenient mode, preserves unparseable fragments as ``Error`` nodes. Since 0.4.0 the default parse configuration is ``LENIENT`` (``reject_unknown=False``, ``abort_on_error=False``). Per-call ``config`` must be a complete ``ParseConfig`` instance; a dict passed as ``config`` raises ``ConfigError``. Other keywords are overlays (``parse(src, reject_unknown=True)``). Construct a ``KnowledgeBase`` to customize packages and records, then share it with parsers and engines. For the conceptual model, see the Parsing guide. ```python parser = texform.Parser() result = parser.parse(r"\frac{x}{y}") result["document"].to_latex() # '\frac { x } { y }' parser.parse(r"\frac{x}{y}", config=texform.ParseConfig(reject_unknown=True)) ``` - [ParseConfig](https://texform.dev/api/python/parser.md#ParseConfig) - [Document](https://texform.dev/api/python/document.md#Document) - [ParseResult](https://texform.dev/api/python/parser.md#ParseResult) - [Parsing](https://texform.dev/docs/parsing.md) #### `Parser` ```python Parser(knowledge_base: KnowledgeBase | None = None, default_parse_config: ParseConfig | None = None) -> None ``` Construct a parser; None uses the process-wide default knowledge base. | Name | Type | Default | Description | | --- | --- | --- | --- | | `knowledge_base` | KnowledgeBase | None | None | | | `default_parse_config` | ParseConfig | None | None | | #### `parse` ```python parse(src: str, config: ParseConfig | None = None, **overrides) -> ParseResult ``` Parse a LaTeX string into a parse result. Parsing never raises on malformed input. The result has exactly three honest states: a hard failure (``document`` is ``None``), a clean parse (a ``Document`` whose ``has_errors()`` is ``False``), or a partial parse (a read-only ``Document`` whose ``has_errors()`` is ``True``). Empty input is a clean parse, not ``None``. | Name | Type | Default | Description | | --- | --- | --- | --- | | `src` | str | — | The LaTeX source string. | | `config` | ParseConfig | None | None | A complete ``ParseConfig``; ``**overrides`` are layered on top. ``None`` uses the parser default. A dict here raises ``ConfigError``. | | `**overrides` | Unpack[ParseOverrides] | — | Keyword overlay from ``ParseOverrides``. Nested values must be dicts, not config class instances. | **Returns** `ParseResult` A ``ParseResult`` dict with two keys: ``document`` (a ``Document`` or ``None``) and ``diagnostics`` (a list of diagnostic dicts). - `ConfigError`: If ``config`` is not a ``ParseConfig``, a dict is passed as ``config``, or an override key or value is invalid. ```python result = texform.Parser().parse(r"\frac{x}{y}") document = result["document"] diagnostics = result["diagnostics"] ``` - [ParseConfig](https://texform.dev/api/python/parser.md#ParseConfig) - [ParseResult](https://texform.dev/api/python/parser.md#ParseResult) #### `default_parse_config` ```python default_parse_config() -> ParseConfig ``` Return a new ``ParseConfig`` holding this parser's defaults. **Returns** `ParseConfig` ### Knowledge-base lookups Query the knowledge the parser loaded. Predicate methods take a single `name`; lookup methods take `name` and a `mode` of `"math"` or `"text"` and return a knowledge-entry dict or `None`. #### `knowledge_base` ```python knowledge_base() -> KnowledgeBase ``` Return the immutable knowledge base shared with parsed documents. **Returns** `KnowledgeBase` ## `ParseConfig` Kind: class ```python ParseConfig(reject_unknown: bool = False, abort_on_error: bool = False, max_group_depth: int = 128) -> None ``` Complete parse configuration. The only accepted ``config=`` type for parse. Since 0.4.0 this is a complete configuration class, not a dict. Passing a dict as ``config`` raises ``ConfigError``; use keyword overlays (``parse(src, reject_unknown=True)``) or construct this class and pass it as ``config``. Constructor defaults match ``LENIENT`` (the parser and engine baseline on both language bindings): ``reject_unknown=False``, ``abort_on_error=False``, ``max_group_depth=128``. ``reject_unknown`` and ``abort_on_error`` are independent: the former decides how unknown names are handled, the latter is a strictness knob for error recovery. Neither is equivalent to a parsed tree's ``has_errors()``. ```python cfg = texform.ParseConfig() cfg.reject_unknown # False cfg.abort_on_error # False texform.Parser().parse(r"x", config=texform.ParseConfig(reject_unknown=True)) ``` - [Parser](https://texform.dev/api/python/parser.md#Parser) #### `reject_unknown` When ``True``, an unknown command or environment becomes a diagnostic; when ``False`` (default), unknown names are preserved as nodes for lenient exploration. #### `abort_on_error` When ``True`` (strict), the parser stops at the first error per item and produces no recovery ``Error`` nodes; when ``False`` (default, lenient), it keeps collecting diagnostics and emits ``Error`` nodes. The max-group-depth guard emits an ``Error`` node regardless of this setting. #### `max_group_depth` The maximum group nesting depth before the parser aborts a group. Defaults to ``128``. #### `ParseConfig` ```python ParseConfig(reject_unknown: bool = False, abort_on_error: bool = False, max_group_depth: int = 128) -> None ``` Construct a parse configuration. | Name | Type | Default | Description | | --- | --- | --- | --- | | `reject_unknown` | bool | False | Whether to reject unknown command and environment names with diagnostics instead of preserving them as nodes. | | `abort_on_error` | bool | False | Whether to stop at the first error per item (strict) instead of collecting diagnostics and emitting ``Error`` nodes (lenient). | | `max_group_depth` | int | 128 | The maximum group nesting depth before aborting. | ## `ParseOverrides` Kind: typedDict Keyword overlays for ``parse``. Omitted keys keep the baseline. These are overlay keys, not a complete ``ParseConfig``. Passing a dict as the ``config`` argument still raises ``ConfigError``; use these keys as keywords (``parse(src, reject_unknown=True)``) or expand a saved dict with ``**overrides``. | Name | Type | Description | | --- | --- | --- | | `reject_unknown` | bool | Overlay for ``ParseConfig.reject_unknown``. | | `abort_on_error` | bool | Overlay for ``ParseConfig.abort_on_error``. | | `max_group_depth` | int | Overlay for ``ParseConfig.max_group_depth``. | ## `ParseResult` Kind: typedDict The result of ``Parser.parse``: an optional document plus diagnostics. There are exactly three honest states, distinguished by ``document`` and the document's ``has_errors()``: a hard failure (``document`` is ``None``), a clean parse (a ``Document`` whose ``has_errors()`` is ``False``), and a partial parse (a read-only ``Document`` whose ``has_errors()`` is ``True``). The parser never fabricates a placeholder tree, and empty input is a clean parse rather than ``None``. | Name | Type | Description | | --- | --- | --- | | `document` | Document | None | The parsed ``Document``, or ``None`` on a hard failure. | | `diagnostics` | list[ParseDiagnostic] | The diagnostics describing what the parser noticed. | - [Parser](https://texform.dev/api/python/parser.md#Parser) - [Document](https://texform.dev/api/python/document.md#Document) - [ParseDiagnostic](https://texform.dev/api/python/parser.md#ParseDiagnostic) ## `ParseDiagnostic` Kind: typedDict A single parse diagnostic emitted alongside a parse result. Diagnostics are a channel separate from ``Error`` placeholder nodes: a diagnostic describes something the parser noticed, while an ``Error`` node is a recovered fragment in the tree. A tree that ``has_errors()`` always carries diagnostics, but the converse does not hold. All fields are optional. | Name | Type | Description | | --- | --- | --- | | `kind` | ParseDiagnosticKind | The diagnostic category, one of the ``ParseDiagnosticKind`` values such as ``"unknown-command"`` or ``"argument-validation"``. | | `message` | str | A human-readable description of the problem. | | `span` | Span | A ``{"start": int, "end": int}`` byte range into the source. | | `expected` | list[str] | The tokens that would have been accepted (may be empty). | | `found` | str | The token actually found, or absent when not applicable. | | `contexts` | list[dict[str, Any]] | Nested context dicts describing where the diagnostic arose (may be empty). | ## `ParseDiagnosticKind` Kind: typeAlias ```python Literal['ambiguous-infix', 'argument-validation', 'command-mode-error', 'comment-truncated-argument', 'environment-mode-error', 'environment-name-mismatch', 'invalid-character', 'left-right-delimiter', 'max-group-depth-exceeded', 'raw-expected-found', 'text-script-error', 'unclosed-environment', 'unclosed-inline-math', 'unexpected-math-shift', 'unknown-command', 'unknown-environment'] ``` Category of a parse diagnostic, such as ``"unknown-command"`` or ``"argument-validation"``. ## `Span` Kind: typeAlias ```python dict[str, int] ``` Half-open UTF-8 byte range ``{"start": int, "end": int}`` into the source. ## `SyntaxNode` Kind: typeAlias ```python dict[str, Any] ``` Lossless parse snapshot as a plain dict. The tagged-union shape is documented on the TypeScript ``SyntaxNode`` type; Python keeps this alias unexpanded. ## `ContextItem` Kind: typeAlias ```python CommandItem | EnvironmentItem | DelimiterItem ``` A knowledge-injection item: a command, environment, or delimiter-control dict. ## `CommandItem` Kind: typedDict A custom command injected into parser knowledge. | Name | Type | Description | | --- | --- | --- | | `target` | Literal['command'] | Discriminator; always ``"command"``. | | `name` | str | Command name without the leading backslash. | | `kind` | Literal['prefix', 'infix', 'declarative'] | Syntactic role: ``"prefix"``, ``"infix"``, or ``"declarative"``. | | `allowed_mode` | Literal['math', 'text', 'both'] | Modes in which the command is valid: ``"math"``, ``"text"``, or ``"both"``. | | `argspec` | str | xparse-style argument specification, such as ``"m m"`` or ``"o m"``. | | `tags` | NotRequired[list[str]] | Optional free-form classification tags carried into the knowledge entry. | ## `EnvironmentItem` Kind: typedDict A custom environment injected into parser knowledge. | Name | Type | Description | | --- | --- | --- | | `target` | Literal['environment'] | Discriminator; always ``"environment"``. | | `name` | str | Environment name as written in ``\begin{...}``. | | `allowed_mode` | Literal['math', 'text', 'both'] | Modes in which the environment may appear. | | `body_mode` | Literal['math', 'text'] | Content mode of the environment body (``"math"`` or ``"text"``). | | `argspec` | str | xparse-style argument specification for the environment header. | | `tags` | NotRequired[list[str]] | Optional free-form classification tags. | ## `DelimiterItem` Kind: typedDict A custom delimiter control injected into parser knowledge. | Name | Type | Description | | --- | --- | --- | | `target` | Literal['delimiter'] | Discriminator; always ``"delimiter"``. | | `name` | str | Delimiter-control name without the leading backslash (for example ``"langle"``). | --- # KnowledgeBase Share immutable knowledge across parsers, transform engines, and documents. https://texform.dev/api/python/knowledge-base ## `KnowledgeBase` Kind: class ```python KnowledgeBase(packages: list[str] | None = None, items: list[ContextItem] | None = None, remove_commands: list[str] | None = None, remove_environments: list[str] | None = None, remove_delimiter_controls: list[str] | None = None) -> None ``` Immutable knowledge shared by parsers, engines, and documents. A knowledge base defines the known commands, environments, characters, and delimiters that parsing, transforms, and ``Document`` construction and edits check against. Share one instance across a ``Parser``, ``TransformEngine``, and their documents. Equality and hashing use instance identity: separately constructed knowledge bases are distinct even with identical recipes. Query results are copies. ``packages=None`` loads the default runtime packages; ``[]`` loads none. ```python kb = texform.KnowledgeBase(["ams"]) doc = texform.Parser(kb).parse(r"\frac{a}{b}")["document"] ``` #### `KnowledgeBase` ```python KnowledgeBase(packages: list[str] | None = None, items: list[ContextItem] | None = None, remove_commands: list[str] | None = None, remove_environments: list[str] | None = None, remove_delimiter_controls: list[str] | None = None) -> None ``` Build the knowledge base once. | Name | Type | Default | Description | | --- | --- | --- | --- | | `packages` | list[str] | None | None | Package names to load, or ``None`` for the defaults. | | `items` | list[ContextItem] | None | None | Extra command, environment, or delimiter items to add. | | `remove_commands` | list[str] | None | None | Command names to remove after loading. | | `remove_environments` | list[str] | None | None | Environment names to remove after loading. | | `remove_delimiter_controls` | list[str] | None | None | Delimiter control names to remove after loading. | - `ConfigError`: If a package name or an item is invalid. #### `__eq__` ```python __eq__(other: object) -> bool ``` | Name | Type | Default | Description | | --- | --- | --- | --- | | `other` | object | — | | **Returns** `bool` #### `__hash__` ```python __hash__() -> int ``` **Returns** `int` #### `packages` ```python packages() -> list[str] ``` Return the loaded package names in canonical order. **Returns** `list[str]` #### `commands` ```python commands(mode: Literal['math', 'text']) -> list[dict[str, Any]] ``` Return the command records for ``mode``, sorted by name. | Name | Type | Default | Description | | --- | --- | --- | --- | | `mode` | Literal['math', 'text'] | — | | **Returns** `list[dict[str, Any]]` #### `environments` ```python environments(mode: Literal['math', 'text']) -> list[dict[str, Any]] ``` Return the environment records for ``mode``, sorted by name. | Name | Type | Default | Description | | --- | --- | --- | --- | | `mode` | Literal['math', 'text'] | — | | **Returns** `list[dict[str, Any]]` #### `characters` ```python characters(mode: Literal['math', 'text']) -> list[dict[str, Any]] ``` Return the character records for ``mode``, sorted by name. | Name | Type | Default | Description | | --- | --- | --- | --- | | `mode` | Literal['math', 'text'] | — | | **Returns** `list[dict[str, Any]]` #### `delimiters` ```python delimiters() -> list[dict[str, Any]] ``` Return delimiter records sorted by name. **Returns** `list[dict[str, Any]]` #### `lookup_command` ```python lookup_command(name: str, mode: Literal['math', 'text']) -> dict[str, Any] | None ``` Look up the full knowledge entry for a command in a given mode. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | str | — | The command name without the leading backslash. | | `mode` | Literal['math', 'text'] | — | ``"math"`` or ``"text"``. | **Returns** `dict[str, Any] | None` A dict describing the command, including its parsed ``args`` slots, or ``None`` if the command is unknown in that mode. #### `lookup_explicit_command` ```python lookup_explicit_command(name: str, mode: Literal['math', 'text']) -> dict[str, Any] | None ``` Look up the knowledge entry for an explicit command in a given mode. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | str | — | The command name without the leading backslash. | | `mode` | Literal['math', 'text'] | — | ``"math"`` or ``"text"``. | **Returns** `dict[str, Any] | None` A knowledge-entry dict, or ``None`` if unknown in that mode. #### `lookup_character` ```python lookup_character(name: str, mode: Literal['math', 'text']) -> dict[str, Any] | None ``` Look up the knowledge entry for a character in a given mode. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | str | — | The character name without the leading backslash. | | `mode` | Literal['math', 'text'] | — | ``"math"`` or ``"text"``. | **Returns** `dict[str, Any] | None` A knowledge-entry dict, or ``None`` if unknown in that mode. #### `lookup_env` ```python lookup_env(name: str, mode: Literal['math', 'text']) -> dict[str, Any] | None ``` Look up the knowledge entry for an environment in a given mode. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | str | — | The environment name. | | `mode` | Literal['math', 'text'] | — | ``"math"`` or ``"text"``. | **Returns** `dict[str, Any] | None` A knowledge-entry dict, or ``None`` if unknown in that mode. #### `is_delimiter_control` ```python is_delimiter_control(name: str) -> bool ``` Report whether a name is a delimiter control such as ``langle``. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | str | — | The control name without the leading backslash. | **Returns** `bool` ``True`` if the name is a known delimiter control. ```python texform.KnowledgeBase().is_delimiter_control("langle") # True ``` #### `knows_command_name` ```python knows_command_name(name: str) -> bool ``` Report whether a command name is known, ignoring mode. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | str | — | The command name without the leading backslash. | **Returns** `bool` ``True`` if the command is known in any mode. #### `knows_env_name` ```python knows_env_name(name: str) -> bool ``` Report whether an environment name is known, ignoring mode. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | str | — | The environment name. | **Returns** `bool` ``True`` if the environment is known in any mode. #### `knows_character_name` ```python knows_character_name(name: str) -> bool ``` Report whether a character name is known, ignoring mode. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | str | — | The character name without the leading backslash. | **Returns** `bool` ``True`` if the character is known in any mode. ## `CommandItem` Kind: typedDict A custom command injected into parser knowledge. | Name | Type | Description | | --- | --- | --- | | `target` | Literal['command'] | Discriminator; always ``"command"``. | | `name` | str | Command name without the leading backslash. | | `kind` | Literal['prefix', 'infix', 'declarative'] | Syntactic role: ``"prefix"``, ``"infix"``, or ``"declarative"``. | | `allowed_mode` | Literal['math', 'text', 'both'] | Modes in which the command is valid: ``"math"``, ``"text"``, or ``"both"``. | | `argspec` | str | xparse-style argument specification, such as ``"m m"`` or ``"o m"``. | | `tags` | NotRequired[list[str]] | Optional free-form classification tags carried into the knowledge entry. | ## `EnvironmentItem` Kind: typedDict A custom environment injected into parser knowledge. | Name | Type | Description | | --- | --- | --- | | `target` | Literal['environment'] | Discriminator; always ``"environment"``. | | `name` | str | Environment name as written in ``\begin{...}``. | | `allowed_mode` | Literal['math', 'text', 'both'] | Modes in which the environment may appear. | | `body_mode` | Literal['math', 'text'] | Content mode of the environment body (``"math"`` or ``"text"``). | | `argspec` | str | xparse-style argument specification for the environment header. | | `tags` | NotRequired[list[str]] | Optional free-form classification tags. | ## `DelimiterItem` Kind: typedDict A custom delimiter control injected into parser knowledge. | Name | Type | Description | | --- | --- | --- | | `target` | Literal['delimiter'] | Discriminator; always ``"delimiter"``. | | `name` | str | Delimiter-control name without the leading backslash (for example ``"langle"``). | --- # Document Construct, query, edit, and serialize the editable TeXForm document tree. https://texform.dev/api/python/document ## `Document` Kind: class ```python Document(knowledge_base: KnowledgeBase | None = None, mode: Literal['math', 'text'] = 'math') -> None ``` The editable document tree — the working format you read, edit, and output. A ``Document`` wraps the internal arena tree behind a fallible, DOM-style surface, so no internal panic ever reaches the caller. Reads go through read-only ``Node`` handles; edits are validated eagerly and raise on misuse. The tree is always structurally valid, even when it contains ``Error`` nodes. A tree that ``has_errors()`` is read-only, fixed that way at construction: every editing method then raises ``EditError``. The only use for an error tree is inspection. Construct one by parsing (``Parser.parse``), from scratch (``Document()``), or from a snapshot (``Document.from_syntax``). For the conceptual model, see the Parsing guide. ```python import texform doc = texform.Parser().parse(r"\frac{x}{y}")["document"] doc.to_latex() # '\frac { x } { y }' ``` - [Node](https://texform.dev/api/python/node.md#Node) - [Parser](https://texform.dev/api/python/parser.md#Parser) - [ParseResult](https://texform.dev/api/python/parser.md#ParseResult) - [Parsing](https://texform.dev/docs/parsing.md) ### Construction and copying #### `Document` ```python Document(knowledge_base: KnowledgeBase | None = None, mode: Literal['math', 'text'] = 'math') -> None ``` Construct an empty document holding a single empty root. This is a complete tree: ``has_errors()`` is ``False`` and it is fully editable. It serializes to the empty string. | Name | Type | Default | Description | | --- | --- | --- | --- | | `knowledge_base` | KnowledgeBase | None | None | | | `mode` | Literal['math', 'text'] | 'math' | | ```python doc = texform.Document() doc.to_latex() # '' ``` #### `from_syntax` ```python from_syntax(node: SyntaxNode, knowledge_base: KnowledgeBase | None = None) -> Document ``` Build a document from a ``SyntaxNode`` dict, the lossless parse snapshot. ``from_syntax`` and ``to_syntax`` are symmetric over every node kind, including ``Error`` and ``Prime``. A bare ``Prime`` is a math symbol; use ``Scripted`` with an empty group base for a leading quote. Invalid external syntax is rejected rather than corrupting the tree. The document can be transformed by any engine sharing its knowledge-base instance. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | SyntaxNode | — | A ``SyntaxNode`` dict, typically produced by ``to_syntax()``. | | `knowledge_base` | KnowledgeBase | None | None | | **Returns** `Document` A new ``Document`` wrapping the snapshot. - `ConformanceError`: If the snapshot does not conform to the knowledge base. ```python result = texform.Parser().parse(r"\frac{x}{y}") document = result["document"] assert document is not None syntax = document.to_syntax() doc = texform.Document.from_syntax(syntax) ``` - [Document.to_syntax](https://texform.dev/api/python/document.md#Document.to_syntax) #### `copy` ```python copy() -> Document ``` Deep-copy the tree into a new document with new node identities. The copy shares this document's knowledge base, so a transform engine built on it accepts both. Also available as ``copy.copy`` and ``copy.deepcopy``. **Returns** `Document` #### `__copy__` ```python __copy__() -> Document ``` **Returns** `Document` #### `__deepcopy__` ```python __deepcopy__(memo: dict[int, Any]) -> Document ``` | Name | Type | Default | Description | | --- | --- | --- | --- | | `memo` | dict[int, Any] | — | | **Returns** `Document` ### Queries #### `root` ```python root() -> Node ``` Return the root ``Node`` of the tree. **Returns** `Node` The ``Root`` node handle. ```python doc.root().kind() # 'Root' ``` #### `has_errors` ```python has_errors() -> bool ``` Report whether the tree contains any ``Error`` placeholder node. This is a cheap O(1) signal, separate from structural validity and independent of the ``abort_on_error`` parse setting. When ``True``, the document is read-only. **Returns** `bool` ``True`` if the tree holds at least one ``Error`` node. ```python result = texform.Parser().parse(r"\sqrt[") document = result["document"] assert document is not None document.has_errors() # True ``` - [Document.errors](https://texform.dev/api/python/document.md#Document.errors) - [Document.is_read_only](https://texform.dev/api/python/document.md#Document.is_read_only) #### `is_read_only` ```python is_read_only() -> bool ``` Report whether the document is read-only. A tree that ``has_errors()`` is read-only and fixed that way at construction; its error count cannot change, so its read-only-ness cannot either. **Returns** `bool` ``True`` if no editing method will succeed on this document. #### `errors` ```python errors() -> list[Node] ``` Return the ``Error`` placeholder nodes in the tree. **Returns** `list[Node]` The list of ``Error`` node handles, empty when ``has_errors()`` is ``False``. #### `find_commands` ```python find_commands(name: str) -> list[Node] ``` Return every command node whose name matches. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | str | — | The command name without the leading backslash. | **Returns** `list[Node]` The matching ``Command`` node handles, in document order. ```python doc = texform.Parser().parse(r"\frac{x}{y} + \frac{a}{b}")["document"] len(doc.find_commands("frac")) # 2 ``` #### `find_environments` ```python find_environments(name: str) -> list[Node] ``` Return every environment node whose name matches. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | str | — | The environment name. | **Returns** `list[Node]` The matching ``Environment`` node handles, in document order. #### `node_spans` ```python node_spans() -> list[NodeSpanEntry] ``` Return source byte spans for the nodes that carry them. **Returns** `list[NodeSpanEntry]` A list of ``{"id": str, "span": {...}}`` entries mapping node identifiers to source byte ranges. #### `node_at` ```python node_at(path: str) -> Node ``` Resolve a tree path to a node handle. Paths are rooted at ``root`` and use the scheme of ``Node.path()``; use paths read from the current tree, since edits shift them. | Name | Type | Default | Description | | --- | --- | --- | --- | | `path` | str | — | A path such as one returned by ``Node.path()``. | **Returns** `Node` The node at that path. - `EditError`: If the path does not exist. #### `knowledge_base` ```python knowledge_base() -> KnowledgeBase ``` Return the immutable knowledge base bound to this document. Construction and edits are checked against it; ``to_latex`` is not. **Returns** `KnowledgeBase` ### Output #### `to_latex` ```python to_latex(**options) -> str ``` Serialize the tree back to LaTeX text. ``Error`` nodes round-trip their captured source snippet verbatim, so a partial tree round-trips losslessly. The serializer guarantees text idempotency: re-parsing and re-serializing the output yields the same string. There is intentionally no method named ``serialize`` on ``Document``; this is the text channel and ``to_syntax()`` is the data channel. | Name | Type | Default | Description | | --- | --- | --- | --- | | `**options` | Unpack[SerializeOptions] | — | ``SerializeOptions`` keywords. Omitted keys and explicit ``None`` keep the default (spaced) style. | **Returns** `str` The serialized LaTeX string. - `ConfigError`: If an option key is unknown or a value has the wrong type. - `TypeError`: If options are passed as a positional dict instead of keyword arguments. Use ``doc.to_latex(**options)`` for a dict. ```python doc = texform.Parser().parse(r"x^2")["document"] doc.to_latex() # 'x ^ { 2 }' doc.to_latex(style="compact") # 'x^2' ``` - [SerializeOptions](https://texform.dev/api/python/serialize.md#SerializeOptions) - [serialize](https://texform.dev/api/python/serialize.md#serialize) - [Document.to_syntax](https://texform.dev/api/python/document.md#Document.to_syntax) #### `to_tokenized_latex` ```python to_tokenized_latex(**options) -> TokenizedLatex ``` Serialize the tree to LaTeX together with typed output tokens. ``start_byte`` and ``end_byte`` are UTF-8 byte offsets, not Python string indices. Empty error snippets produce no zero-width token; use ``has_errors()`` to detect whether the document contains error nodes. | Name | Type | Default | Description | | --- | --- | --- | --- | | `**options` | Unpack[SerializeOptions] | — | ``SerializeOptions`` keywords, as for ``to_latex``. Omitted keys and explicit ``None`` keep the default (spaced) style. | **Returns** `TokenizedLatex` A ``TokenizedLatex`` dict with ``latex`` and ``tokens``. - `ConfigError`: If an option key is unknown or a value has the wrong type. ```python tok = texform.Parser().parse(r"x^2")["document"].to_tokenized_latex() tok["latex"] # 'x ^ { 2 }' tok["tokens"][0]["kind"] # 'character' ``` #### `to_syntax` ```python to_syntax() -> SyntaxNode ``` Convert the tree to a ``SyntaxNode`` dict for serde and transport. This is the structured-data channel, distinct from the text channel ``to_latex()``. The conversion is symmetric with ``from_syntax``. **Returns** `SyntaxNode` A ``SyntaxNode`` dict, the single serde DTO for the tree. ```python doc.to_syntax() # {'Root': {...}} ``` - [Document.from_syntax](https://texform.dev/api/python/document.md#Document.from_syntax) - [Document.to_latex](https://texform.dev/api/python/document.md#Document.to_latex) #### `to_columnar` ```python to_columnar() -> ColumnarTree ``` Flatten the tree into columns for bulk structural analysis with Arrow or DataFrame tools. Each table can be passed directly to ``pyarrow.table``. Parent, slot index, and content columns use -1 when no row or index applies. Other missing values are None. Modes describe each node's context. Detached subtrees are excluded; incomplete documents remain exportable. **Returns** `ColumnarTree` ### Staging constructors The `create_*` methods, `parse_fragment`, `clone_node`, and `import_node` build detached subtrees owned by this document. A staged node is not in the tree until you attach it with an edit method. #### `create_char` ```python create_char(value: str, mode: RuntimeContentMode | None = None) -> Node ``` Create a detached ``Char`` node owned by this document. Staged nodes are not in the tree until attached with an edit method. The context ``mode`` defaults to the document root's mode; every ``create_*`` method accepts it as a keyword to construct in another context. | Name | Type | Default | Description | | --- | --- | --- | --- | | `value` | str | — | The single character. ``"&"`` is a literal ampersand written ``\&``. | | `mode` | RuntimeContentMode | None | None | The context mode, ``"math"`` or ``"text"``. | **Returns** `Node` The detached ``Char`` node handle. - `ConformanceError`: If the character cannot be written in that mode, such as ``\``, ``^``, ``~``, or a math-mode space. - `EditError`: If the document is read-only. #### `create_text` ```python create_text(value: str, mode: RuntimeContentMode | None = None) -> Node ``` Create a detached ``Text`` node owned by this document. | Name | Type | Default | Description | | --- | --- | --- | --- | | `value` | str | — | The text value. | | `mode` | RuntimeContentMode | None | None | The context mode. Defaults to the root mode. | **Returns** `Node` The detached ``Text`` node handle. - `ConformanceError`: If the text cannot be written in that mode. - `EditError`: If the document is read-only. #### `create_active_space` ```python create_active_space(mode: RuntimeContentMode | None = None) -> Node ``` Create a detached ``ActiveSpace`` node owned by this document. | Name | Type | Default | Description | | --- | --- | --- | --- | | `mode` | RuntimeContentMode | None | None | | **Returns** `Node` The detached ``ActiveSpace`` node handle. #### `create_group` ```python create_group(mode: RuntimeContentMode | None = None, children: list[Node | str] | None = None) -> Node ``` Create a detached brace ``Group``, built in its own ``mode``. | Name | Type | Default | Description | | --- | --- | --- | --- | | `mode` | RuntimeContentMode | None | None | The group's content mode and context. Defaults to the root mode. | | `children` | list[Node | str] | None | None | Detached nodes or source strings. A string is parsed in ``mode`` and its nodes are spliced into the list, so one string may yield several children. | **Returns** `Node` The detached ``Group`` node handle. - `ParseError`: If a source string does not parse cleanly. - `ConformanceError`: If a child was built in another mode or an infix has siblings. - `EditError`: If a child is attached, foreign, or repeated, or the document is read-only. ```python doc = texform.Document() group = doc.create_group("math", ["x^2", doc.create_char("y")]) ``` #### `create_command` ```python create_command(name: str, args: list[Arg] | None = None, mode: RuntimeContentMode | None = None) -> Node ``` Create a detached prefix ``Command`` owned by this document. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | str | — | The command name without the leading backslash. A name missing from the knowledge base creates an unknown command without arguments. | | `args` | list[Arg] | None | None | Values for every signature slot, or only the required slots. A ``str`` is the source written inside the slot's boundaries; ``None`` omits an optional slot and means false in a star slot; ``Paired`` selects an explicit delimiter pair. ``bool`` sets a star slot. | | `mode` | RuntimeContentMode | None | None | The context mode. Defaults to the root mode. | **Returns** `Node` The detached ``Command`` node handle. - `ConformanceError`: If the name is not a prefix command (the message names the right constructor, such as ``create_declarative`` for ``bf``), the argument count is wrong, or an argument does not fit its slot. - `ParseError`: If a source argument does not parse cleanly. - `EditError`: For invalid nodes or a read-only document. ```python doc = texform.Document() cmd = doc.create_command("sqrt", ["x"]) doc.append_child(doc.root(), cmd) doc.to_latex() # '\sqrt { x }' ``` #### `create_declarative` ```python create_declarative(name: str, args: list[Arg] | None = None, mode: RuntimeContentMode | None = None) -> Node ``` Create a detached ``Declarative`` command such as ``\bf``. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | str | — | The declarative command name without the leading backslash. | | `args` | list[Arg] | None | None | Slot values, as for ``create_command``. | | `mode` | RuntimeContentMode | None | None | The context mode. Defaults to the root mode. | **Returns** `Node` The detached ``Declarative`` node handle. - `ConformanceError`: Under the conditions of ``create_command``, except the record must be declarative. - `ParseError`: If a source argument does not parse cleanly. #### `create_environment` ```python create_environment(name: str, args: list[Arg] | None = None, body: Node | list[Node | str] | str | None = None, mode: RuntimeContentMode | None = None) -> Node ``` Create a detached ``Environment`` owned by this document. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | str | — | The environment name. An unknown environment takes no arguments and uses the surrounding mode. | | `args` | list[Arg] | None | None | Slot values, as for ``create_command``. | | `body` | Node | list[Node | str] | str | None | None | A detached group in the body mode, a source string parsed in the recorded body mode, a list whose source strings are spliced as in ``create_group``, or ``None`` for an empty body. | | `mode` | RuntimeContentMode | None | None | The context mode. Defaults to the root mode. | **Returns** `Node` The detached ``Environment`` node handle. - `ConformanceError`: If an argument does not fit the signature or the body is not a group in the body mode. - `ParseError`: If source does not parse cleanly. - `EditError`: For invalid nodes or a read-only document. ```python doc = texform.Document() env = doc.create_environment("matrix", body="a & b") ``` #### `create_alignment_tab` ```python create_alignment_tab(mode: RuntimeContentMode | None = None) -> Node ``` Create a detached ``AlignmentTab`` node owned by this document. This is the unescaped ``&`` that separates cells in alignment environments. Use ``create_char("&")`` for a literal ampersand, which serializes as ``\&``. | Name | Type | Default | Description | | --- | --- | --- | --- | | `mode` | RuntimeContentMode | None | None | | **Returns** `Node` The detached ``AlignmentTab`` node handle. #### `create_delimited_group` ```python create_delimited_group(left: str, right: str, children: list[Node | str] | None = None, mode: RuntimeContentMode | None = None) -> Node ``` Create a detached ``\left...\right`` group, built in math mode. | Name | Type | Default | Description | | --- | --- | --- | --- | | `left` | str | — | The left delimiter source, such as ``"("`` or ``"\langle"``. | | `right` | str | — | The right delimiter source. | | `children` | list[Node | str] | None | None | Detached nodes or source strings, spliced as in ``create_group``. | | `mode` | RuntimeContentMode | None | None | The context mode. Defaults to ``"math"``. | **Returns** `Node` The detached delimited ``Group`` node handle. - `ConformanceError`: If a delimiter is neither ``.`` nor registered in the knowledge base, or a child does not fit. - `ParseError`: If a source string does not parse cleanly. - `EditError`: For invalid children or a read-only document. #### `create_inline_math` ```python create_inline_math(children: list[Node | str] | None = None, mode: RuntimeContentMode | None = None) -> Node ``` Create a detached ``$...$`` group whose children are math. | Name | Type | Default | Description | | --- | --- | --- | --- | | `children` | list[Node | str] | None | None | Detached nodes or source strings, spliced as in ``create_group``. | | `mode` | RuntimeContentMode | None | None | The context mode. Defaults to ``"text"``. | **Returns** `Node` The detached inline-math ``Group`` node handle. - `ParseError`: If a source string does not parse cleanly. - `ConformanceError`: If a child does not fit or the context is math. - `EditError`: For invalid children or a read-only document. #### `create_scripted` ```python create_scripted(base: Node | str, sub: Node | str | None = None, sup: Node | str | None = None, mode: RuntimeContentMode | None = None) -> Node ``` Create a detached ``Scripted`` node with a base and subscript and/or superscript. | Name | Type | Default | Description | | --- | --- | --- | --- | | `base` | Node | str | — | The base node or source string. | | `sub` | Node | str | None | None | The subscript node or source, or ``None`` for none. | | `sup` | Node | str | None | None | The superscript node or source, or ``None`` for none. | | `mode` | RuntimeContentMode | None | None | The context mode. Defaults to ``"math"``. | **Returns** `Node` The detached ``Scripted`` node handle. - `ConformanceError`: If both scripts are absent, or the base is a scripted node, an infix, or an alignment tab. - `ParseError`: If a source string does not parse cleanly. - `EditError`: For invalid nodes or a read-only document. ```python doc = texform.Document() doc.append_child(doc.root(), doc.create_scripted("x", sup="2")) ``` #### `create_prime` ```python create_prime(count: int, mode: RuntimeContentMode | None = None) -> Node ``` Create a detached run of ``count`` primes. | Name | Type | Default | Description | | --- | --- | --- | --- | | `count` | int | — | The number of primes; must be positive. | | `mode` | RuntimeContentMode | None | None | The context mode. Defaults to ``"math"``. | **Returns** `Node` The detached ``Prime`` node handle. - `ConformanceError`: If ``count`` is zero. #### `create_infix` ```python create_infix(name: str, left: Node | str, right: Node | str, args: list[Arg] | None = None, mode: RuntimeContentMode | None = None) -> Node ``` Create a detached ``Infix`` command such as ``\over``. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | str | — | The infix command name without the leading backslash. | | `left` | Node | str | — | The left operand node or source string. | | `right` | Node | str | — | The right operand node or source string. | | `args` | list[Arg] | None | None | Values for extra slots, such as the dimension of ``\above``. | | `mode` | RuntimeContentMode | None | None | The context mode. Defaults to ``"math"``. | **Returns** `Node` The detached ``Infix`` node handle. - `ConformanceError`: Under the conditions of ``create_command``, except the record must be infix. - `ParseError`: If a source operand or argument does not parse cleanly. #### `parse_fragment` ```python parse_fragment(source: str, mode: RuntimeContentMode | None = None) -> Node ``` Parse source into a detached implicit ``Group``. | Name | Type | Default | Description | | --- | --- | --- | --- | | `source` | str | — | The LaTeX source. | | `mode` | RuntimeContentMode | None | None | The parse mode. Defaults to the root mode. | **Returns** `Node` The detached ``Group`` node handle. - `ParseError`: If the source does not parse cleanly. ``diagnostics`` carries the parser diagnostics. #### `clone_node` ```python clone_node(node: Node) -> Node ``` Deep-copy a subtree of this document as a detached node. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | The subtree root to copy. | **Returns** `Node` The detached copy, ready to attach. - `EditError`: If ``node`` is foreign or missing, or the document is read-only. #### `import_node` ```python import_node(node: Node) -> Node ``` Copy a subtree from another document into this one as a detached node. The subtree is validated against this document's knowledge base, so importing across knowledge bases can fail where ``clone_node`` cannot. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | A node of another document. | **Returns** `Node` The detached copy, owned by this document. - `ConformanceError`: If the subtree does not conform to this knowledge base. - `EditError`: If the edit is invalid. ### Edit methods All edits are validated eagerly and raise `EditError` on misuse. Nodes must belong to this document, the root cannot be detached, and a tree that `has_errors()` is read-only. #### `append_child` ```python append_child(parent: Node, child: Node) -> None ``` Append ``child`` as the last child of ``parent``. | Name | Type | Default | Description | | --- | --- | --- | --- | | `parent` | Node | — | The container node to append into. | | `child` | Node | — | The detached or detached node to attach. | **Returns** `None` - `EditError`: If the edit is invalid (read-only tree, root protection, wrong container shape, or a foreign node). #### `insert_before` ```python insert_before(anchor: Node, new: Node) -> None ``` Insert ``new`` immediately before ``anchor`` among its siblings. | Name | Type | Default | Description | | --- | --- | --- | --- | | `anchor` | Node | — | The node to insert before. | | `new` | Node | — | The detached or detached node to attach. | **Returns** `None` - `EditError`: If the edit is invalid. #### `insert_after` ```python insert_after(anchor: Node, new: Node) -> None ``` Insert ``new`` immediately after ``anchor`` among its siblings. | Name | Type | Default | Description | | --- | --- | --- | --- | | `anchor` | Node | — | The node to insert after. | | `new` | Node | — | The detached or detached node to attach. | **Returns** `None` - `EditError`: If the edit is invalid. #### `insert_child` ```python insert_child(parent: Node, index: int, child: Node) -> None ``` Insert ``child`` at ``index`` among ``parent``'s children. | Name | Type | Default | Description | | --- | --- | --- | --- | | `parent` | Node | — | The container node to insert into. | | `index` | int | — | The zero-based position to insert at. | | `child` | Node | — | The detached or detached node to attach. | **Returns** `None` - `EditError`: If the edit is invalid, including an out-of-bounds index. #### `replace_with` ```python replace_with(target: Node, replacement: Node) -> None ``` Replace ``target`` in the tree with ``replacement``. | Name | Type | Default | Description | | --- | --- | --- | --- | | `target` | Node | — | The node to remove from its position. | | `replacement` | Node | — | The detached or detached node to put in its place. | **Returns** `None` - `EditError`: If the edit is invalid. #### `wrap` ```python wrap(target: Node, wrapper: Node) -> Node ``` Wrap ``target`` inside ``wrapper``, putting ``wrapper`` in its place. | Name | Type | Default | Description | | --- | --- | --- | --- | | `target` | Node | — | The node to be wrapped. | | `wrapper` | Node | — | The detached container node to wrap it in. | **Returns** `Node` The ``wrapper`` node handle, now in the tree. - `EditError`: If the edit is invalid. #### `unwrap` ```python unwrap(group: Node) -> list[Node] ``` Remove a group, splicing its children into the parent in its place. | Name | Type | Default | Description | | --- | --- | --- | --- | | `group` | Node | — | The group node to dissolve. | **Returns** `list[Node]` The freed child node handles, now attached to the parent. - `EditError`: If the edit is invalid. ```python doc = texform.Parser().parse(r"{x y}")["document"] group = next(n for n in doc.root().descendants() if n.kind() == "Group") doc.unwrap(group) doc.to_latex() # 'x y' ``` #### `extract` ```python extract(node: Node) -> Node ``` Detach ``node`` from the tree and return it as a detached node. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | The node to remove from its position and keep. | **Returns** `Node` The detached node handle, reusable in a later edit. - `EditError`: If the edit is invalid. #### `remove` ```python remove(node: Node) -> None ``` Remove ``node`` from the tree and discard it. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | The node to remove. | **Returns** `None` - `EditError`: If the edit is invalid, such as removing the root. #### `clear` ```python clear(container: Node) -> None ``` Remove every child of ``container``. | Name | Type | Default | Description | | --- | --- | --- | --- | | `container` | Node | — | The node whose children are removed. | **Returns** `None` - `EditError`: If the edit is invalid. #### `set_command_name` ```python set_command_name(node: Node, name: str) -> None ``` Rename a command node in place. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | The ``Command`` or ``Declarative`` node to rename. | | `name` | str | — | The new command name without the leading backslash. | **Returns** `None` - `EditError`: If the edit is invalid. #### `set_env_name` ```python set_env_name(node: Node, name: str) -> None ``` Rename an environment, keeping its arguments and body. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | The ``Environment`` node to rename. | | `name` | str | — | The new environment name. | **Returns** `None` - `ConformanceError`: If the new name's signature or body mode does not fit the existing arguments and body. - `EditError`: If the edit is invalid. #### `set_text` ```python set_text(node: Node, value: str) -> None ``` Set a text node's value in place. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | The ``Text`` node to update. | | `value` | str | — | The new text value. | **Returns** `None` - `EditError`: If the edit is invalid. #### `set_char` ```python set_char(node: Node, value: str) -> None ``` Set a char node's value in place. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | The ``Char`` node to update. | | `value` | str | — | The new single character. | **Returns** `None` - `EditError`: If the edit is invalid. #### `set_arg` ```python set_arg(node: Node, index: int, value: Arg) -> None ``` Set the argument at ``index`` of a command-like node or environment. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | The command-like node or environment whose argument is set. | | `index` | int | — | The zero-based slot index. | | `value` | Arg | — | The slot source (a ``str``), content node, star flag, ``None`` to clear an optional slot, or ``Paired``. | **Returns** `None` - `ConformanceError`: If the value does not fit the slot. - `ParseError`: If a source string does not parse cleanly. - `EditError`: If the edit is invalid, including an out-of-bounds index. #### `set_arg_delimiters` ```python set_arg_delimiters(node: Node, index: int, open: str, close: str) -> None ``` Change only the boundaries of a filled paired argument. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | The command-like node. | | `index` | int | — | The zero-based slot index. | | `open` | str | — | The new opening delimiter source. | | `close` | str | — | The new closing delimiter source. | **Returns** `None` - `ConformanceError`: If the slot is not a filled paired slot or the delimiters are not allowed. - `EditError`: If the edit is invalid. #### `set_delimiters` ```python set_delimiters(node: Node, left: str, right: str) -> None ``` Change the delimiters of a ``\left...\right`` group. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | The delimited ``Group`` node. | | `left` | str | — | The new left delimiter source. | | `right` | str | — | The new right delimiter source. | **Returns** `None` - `ConformanceError`: If a delimiter is neither ``.`` nor registered in the knowledge base. - `EditError`: If the edit is invalid. #### `set_subscript` ```python set_subscript(target: Node, sub: Arg) -> Node ``` Set or clear the subscript of ``target``. If ``target`` is not scripted, it becomes the base of a new ``Scripted`` node; an existing scripted base is resolved automatically. | Name | Type | Default | Description | | --- | --- | --- | --- | | `target` | Node | — | The base node or an existing ``Scripted`` node. | | `sub` | Arg | — | The new subscript node or source, or ``None`` to clear it. | **Returns** `Node` The resulting ``Scripted`` node, or the base when clearing the last script collapses the wrapper. - `ConformanceError`: If the base cannot carry scripts. - `ParseError`: If a source string does not parse cleanly. - `EditError`: If the edit is invalid. #### `set_superscript` ```python set_superscript(target: Node, sup: Arg) -> Node ``` Set or clear the superscript of ``target``. Behaves like ``set_subscript`` for the superscript slot. | Name | Type | Default | Description | | --- | --- | --- | --- | | `target` | Node | — | The base node or an existing ``Scripted`` node. | | `sup` | Arg | — | The new superscript node or source, or ``None`` to clear it. | **Returns** `Node` The resulting ``Scripted`` node, or the base after a collapse. - `ConformanceError`: If the base cannot carry scripts. - `ParseError`: If a source string does not parse cleanly. - `EditError`: If the edit is invalid. #### `set_prime_count` ```python set_prime_count(node: Node, count: int) -> None ``` Set the number of primes in a ``Prime`` node. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | The ``Prime`` node. | | `count` | int | — | The new count; must be positive. | **Returns** `None` - `ConformanceError`: If ``count`` is not positive. - `EditError`: If the edit is invalid. ## `NodeSpanEntry` Kind: typedDict One entry mapping a node identifier to its source byte span. | Name | Type | Description | | --- | --- | --- | | `id` | str | The node identifier within the document. | | `span` | Span | A ``{"start": int, "end": int}`` byte range into the source. | ## `TokenizedLatex` Kind: typedDict Serialized LaTeX and tokens produced by the same serializer traversal. Text and tokenized output share one traversal and spacing policy; this is not a second lexer pass. | Name | Type | Description | | --- | --- | --- | | `latex` | str | The serialized LaTeX string, identical to ``Document.to_latex()``. | | `tokens` | list[SerializationToken] | Ordered, non-overlapping, non-empty fragments. Gaps between spans contain serializer-inserted spacing. | ## `SerializationToken` Kind: typedDict One serialization fragment with UTF-8 byte offsets. ``start_byte`` / ``end_byte`` index the accompanying ``TokenizedLatex.latex`` string and are UTF-8 byte offsets, not Python code-point indices. Spans do not overlap. Empty error snippets emit no token. | Name | Type | Description | | --- | --- | --- | | `text` | str | The UTF-8 decoding of the byte slice ``latex.encode("utf-8")[start_byte:end_byte]``. | | `start_byte` | int | Inclusive UTF-8 byte offset into the serialized LaTeX string. | | `end_byte` | int | Exclusive UTF-8 byte offset into the serialized LaTeX string. | | `kind` | SerializationTokenKind | Semantic category of the fragment (control sequence, character, ...). | | `mode` | RuntimeContentMode | Semantic math/text mode of the fragment, independent of spacing policy. | ## `SerializationTokenKind` Kind: typeAlias ```python Literal['control_sequence', 'character', 'delimiter', 'text', 'raw', 'error'] ``` Semantic category of one serialization fragment. ## `NodeTable` Kind: typedDict Preorder node columns. Row indices are local to this snapshot. | Name | Type | Description | | --- | --- | --- | | `kind` | list[NodeKind] | | | `parent` | list[int] | | | `slot` | list[NodeSlotKind | None] | | | `slot_index` | list[int] | | | `depth` | list[int] | | | `mode` | list[RuntimeContentMode] | | | `name` | list[str | None] | | | `value` | list[str | None] | | | `known` | list[bool | None] | | | `group_kind` | list[Literal['explicit', 'implicit', 'delimited', 'inline_math'] | None] | | | `left` | list[str | None] | | | `right` | list[str | None] | | ## `ArgumentTable` Kind: typedDict All argument slots, ordered by owner row and slot index. | Name | Type | Description | | --- | --- | --- | | `owner` | list[int] | | | `index` | list[int] | | | `form` | list[ArgFormKind | None] | | | `present` | list[bool] | | | `value_kind` | list[Literal['math', 'text', 'operator_name', 'delimiter', 'cs_name', 'dimension', 'integer', 'key_val', 'column', 'boolean'] | None] | | | `value` | list[str | None] | | | `content` | list[int] | | | `open` | list[str | None] | | | `close` | list[str | None] | | ## `ColumnarTree` Kind: typedDict Tree columns for bulk structural analysis with Arrow or DataFrame tools. | Name | Type | Description | | --- | --- | --- | | `nodes` | NodeTable | | | `args` | ArgumentTable | | --- # Node Navigate the document tree and read node content through read-only Node handles. https://texform.dev/api/python/node ## `Node` Kind: class A read-only handle into a ``Document`` for navigation and reading. A ``Node`` is a cheap reference — a shared pointer into the owning document plus an identity, never a copy of the subtree. It carries no editing methods; all mutation goes through the owning ``Document``. Accessors return content only for the node kinds that carry it and ``None`` otherwise, so the same handle works uniformly across kinds. You obtain handles from ``Document.root()``, ``Document.errors()``, ``Document.find_commands()``, the ``create_*`` staging constructors, and the navigation methods here. ```python doc = texform.Parser().parse(r"\frac{x}{y}")["document"] cmd = doc.root().children()[0] cmd.kind() # 'Command' cmd.is_command("frac") # True ``` - [Document](https://texform.dev/api/python/document.md#Document) ### Kind #### `kind` ```python kind() -> NodeKind ``` Return the node kind as a string. **Returns** `NodeKind` One of the ``NodeKind`` values, such as ``"Root"``, ``"Command"``, or ``"Char"``. ```python doc = texform.Parser().parse(r"\frac{x}{y}")["document"] doc.root().kind() # 'Root' ``` ### Predicates #### `is_command` ```python is_command(name: str | None = None) -> bool ``` Report whether the node is a command, optionally a specific one. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | str | None | None | A command name to match, or ``None`` to match any command. | **Returns** `bool` ``True`` if the node is a command (and matches ``name`` when given). ```python node.is_command() # any command node.is_command("frac") # specifically \frac ``` #### `is_char` ```python is_char(value: str | None = None) -> bool ``` Report whether the node is a char, optionally a specific one. | Name | Type | Default | Description | | --- | --- | --- | --- | | `value` | str | None | None | A character to match, or ``None`` to match any char. | **Returns** `bool` ``True`` if the node is a char (and holds ``value`` when given). ```python node.is_char("+") ``` #### `is_error` ```python is_error() -> bool ``` Report whether the node is an ``Error`` placeholder. **Returns** `bool` ``True`` for an ``Error`` node. #### `is_known` ```python is_known() -> bool | None ``` Report whether the document's knowledge base knows this command or environment. **Returns** `bool | None` ``True`` or ``False`` for command-like nodes and environments, ``None`` for other kinds. ### Navigation #### `parent` ```python parent() -> Node | None ``` Return the parent node, or ``None`` for the root. **Returns** `Node | None` The parent handle, or ``None``. #### `children` ```python children() -> list[Node] ``` Return the node's direct children. **Returns** `list[Node]` The child handles in order. ```python doc = texform.Parser().parse(r"\frac{x}{y} + a")["document"] [n.kind() for n in doc.root().children()] # ['Command', 'Char', 'Char'] ``` #### `next_sibling` ```python next_sibling() -> Node | None ``` Return the following sibling, or ``None`` if this is the last child. **Returns** `Node | None` The next sibling handle, or ``None``. #### `prev_sibling` ```python prev_sibling() -> Node | None ``` Return the preceding sibling, or ``None`` if this is the first child. **Returns** `Node | None` The previous sibling handle, or ``None``. #### `ancestors` ```python ancestors() -> list[Node] ``` Return the node's ancestors from the parent upward. **Returns** `list[Node]` The ancestor handles, nearest first. #### `descendants` ```python descendants() -> list[Node] ``` Return every descendant of the node. **Returns** `list[Node]` The descendant handles in document order. #### `path` ```python path() -> str | None ``` Return the node's current path from the root, or ``None`` if detached. The path reflects the tree now and changes after edits; ``Document.node_at`` resolves it. **Returns** `str | None` #### `slot` ```python slot() -> NodeSlot | None ``` Return the slot this node fills in its parent, or ``None`` for the root and detached nodes. **Returns** `NodeSlot | None` A dict with the slot ``kind`` and, for indexed slots such as children and arguments, its ``index``. ### Content accessors #### `command_name` ```python command_name() -> str | None ``` Return the command name for a ``Command`` / ``Declarative`` node. **Returns** `str | None` The name without the leading backslash, or ``None`` for other kinds. #### `env_name` ```python env_name() -> str | None ``` Return the environment name for an ``Environment`` node. **Returns** `str | None` The environment name, or ``None`` for other kinds. #### `text` ```python text() -> str | None ``` Return the value of a ``Text`` node. **Returns** `str | None` The text string, or ``None`` for other kinds. #### `char` ```python char() -> str | None ``` Return the character of a ``Char`` node. **Returns** `str | None` The single character, or ``None`` for other kinds. #### `prime_count` ```python prime_count() -> int | None ``` Return the symbol count of a math ``Prime`` node. ``Scripted`` determines whether these symbols form a superscript. **Returns** `int | None` The count (greater than zero), or ``None`` for other kinds. ```python doc = texform.Parser().parse(r"f''")["document"] prime = next(n for n in doc.root().descendants() if n.kind() == "Prime") prime.prime_count() # 2 ``` #### `error_parts` ```python error_parts() -> ErrorParts | None ``` Return the decomposed content of an ``Error`` node. **Returns** `ErrorParts | None` A dict ``{"message": str, "snippet": str}``, or ``None`` for other kinds. ```python doc = texform.Parser().parse(r"\sqrt[")["document"] err = next(n for n in doc.root().descendants() if n.is_error()) err.error_parts() # {'message': '...', 'snippet': '\sqrt['} ``` #### `content_mode` ```python content_mode() -> RuntimeContentMode | None ``` Return the runtime content mode where applicable. **Returns** `RuntimeContentMode | None` ``"math"`` or ``"text"`` for nodes that carry a mode, or ``None``. ### Groups #### `group_kind` ```python group_kind() -> GroupKindRef | None ``` Return the group kind for a ``Group`` node. **Returns** `GroupKindRef | None` A dict with a ``kind`` discriminator (``Explicit``, ``Implicit``, ``InlineMath``, or ``Delimited``); a ``Delimited`` group also carries ``left`` and ``right`` delimiter values. ``None`` for other kinds. ```python doc = texform.Parser().parse(r"{x}")["document"] group = next(n for n in doc.root().descendants() if n.kind() == "Group") group.group_kind() # {'kind': 'Explicit'} ``` ### Command arguments #### `arg_count` ```python arg_count() -> int ``` Return the number of argument slots for a command-like node. **Returns** `int` The slot count (zero for nodes that take no arguments). #### `arg` ```python arg(index: int) -> ArgRef | None ``` Return the argument at ``index``, or ``None`` if the slot is empty. | Name | Type | Default | Description | | --- | --- | --- | --- | | `index` | int | — | The zero-based slot index. | **Returns** `ArgRef | None` An argument-value dict — content kinds (``Math``, ``Text``) carry a live ``node``; leaf kinds carry a ``value`` — or ``None`` for an empty slot. ```python doc = texform.Parser().parse(r"\frac{x}{y}")["document"] frac = next(n for n in doc.root().descendants() if n.is_command("frac")) frac.arg_count() # 2 frac.arg(0) # {'kind': 'Math', 'node': } ``` #### `arg_slots` ```python arg_slots() -> list[ArgRef | None] ``` Return all argument slots, with ``None`` for any empty slot. **Returns** `list[ArgRef | None]` The list of argument-value dicts and ``None`` placeholders. ### Scripts and infix #### `script_base` ```python script_base() -> Node | None ``` Return the base of a ``Scripted`` node. **Returns** `Node | None` The base node handle, or ``None`` for other kinds. #### `subscript` ```python subscript() -> Node | None ``` Return the subscript slot of a ``Scripted`` node. **Returns** `Node | None` The subscript node handle, or ``None`` if absent or another kind. ```python doc = texform.Parser().parse(r"x_i^2")["document"] sc = next(n for n in doc.root().descendants() if n.kind() == "Scripted") sc.subscript().kind(), sc.superscript().kind() # ('Char', 'Char') ``` #### `superscript` ```python superscript() -> Node | None ``` Return the superscript slot of a ``Scripted`` node. **Returns** `Node | None` The superscript node handle, or ``None`` if absent or another kind. #### `infix_left` ```python infix_left() -> Node | None ``` Return the left operand of an ``Infix`` node. **Returns** `Node | None` The left operand handle, or ``None`` for other kinds. #### `infix_right` ```python infix_right() -> Node | None ``` Return the right operand of an ``Infix`` node. **Returns** `Node | None` The right operand handle, or ``None`` for other kinds. #### `env_body` ```python env_body() -> Node | None ``` Return the body group of an ``Environment`` node. **Returns** `Node | None` The body node handle, or ``None`` for other kinds. ### Spans #### `span` ```python span() -> Span | None ``` Return the source byte span of the node. **Returns** `Span | None` A ``{"start": int, "end": int}`` dict, or ``None`` if the node has no recorded span. ```python doc = texform.Parser().parse(r"a + b")["document"] doc.root().children()[0].span() # {'start': 0, 'end': 1} ``` ### Identity #### `document` ```python document() -> Document ``` Return the ``Document`` that owns this handle. **Returns** `Document` #### `__eq__` ```python __eq__(other: object) -> bool ``` | Name | Type | Default | Description | | --- | --- | --- | --- | | `other` | object | — | | **Returns** `bool` #### `__hash__` ```python __hash__() -> int ``` **Returns** `int` #### `__repr__` ```python __repr__() -> str ``` **Returns** `str` ## `NodeKind` Kind: typeAlias ```python Literal['Root', 'Group', 'Command', 'Infix', 'Declarative', 'Environment', 'Scripted', 'Prime', 'Text', 'Char', 'ActiveSpace', 'AlignmentTab', 'Error'] ``` Node kind as reported by ``Node.kind()``. Python uses PascalCase (``"Root"``, ``"Command"``, ``"ActiveSpace"``). ## `RuntimeContentMode` Kind: typeAlias ```python Literal['math', 'text'] ``` Runtime content mode reported by the tree API: ``"math"`` or ``"text"``. ## `ErrorParts` Kind: typedDict The decomposed content of an ``Error`` placeholder node. | Name | Type | Description | | --- | --- | --- | | `message` | str | The reason the fragment failed to parse. | | `snippet` | str | The captured source text, re-emitted verbatim on serialization. | ## `GroupKindExplicit` Kind: typedDict A group written with explicit braces ``{ ... }``. | Name | Type | Description | | --- | --- | --- | | `kind` | Literal['Explicit'] | Discriminator; always ``"Explicit"``. | ## `GroupKindImplicit` Kind: typedDict A group with no surrounding braces, inferred from structure. | Name | Type | Description | | --- | --- | --- | | `kind` | Literal['Implicit'] | Discriminator; always ``"Implicit"``. | ## `GroupKindDelimited` Kind: typedDict A group bounded by a delimiter pair, such as ``\left( ... \right)``. | Name | Type | Description | | --- | --- | --- | | `kind` | Literal['Delimited'] | Discriminator; always ``"Delimited"``. | | `left` | DelimiterValue | The opening delimiter value. | | `right` | DelimiterValue | The closing delimiter value. | ## `GroupKindInlineMath` Kind: typedDict A group introduced by inline math shift, such as ``$ ... $`` in text mode. | Name | Type | Description | | --- | --- | --- | | `kind` | Literal['InlineMath'] | Discriminator; always ``"InlineMath"``. | ## `GroupKindRef` Kind: typeAlias ```python GroupKindExplicit | GroupKindImplicit | GroupKindDelimited | GroupKindInlineMath ``` The kind of a ``Group`` node: explicit, implicit, delimited, or inline-math. ## `DelimiterValue` Kind: typeAlias ```python str ``` A delimiter string: a character, a backslash-prefixed control, or a dot. ## `MathArg` Kind: typedDict A command argument carrying math-mode content. | Name | Type | Description | | --- | --- | --- | | `kind` | Literal['Math'] | Discriminator; always ``"Math"``. node: The live argument-body ``Node``. | | `node` | Node | | ## `TextArg` Kind: typedDict A command argument carrying text-mode content. | Name | Type | Description | | --- | --- | --- | | `kind` | Literal['Text'] | Discriminator; always ``"Text"``. node: The live argument-body ``Node``. | | `node` | Node | | ## `DelimiterArg` Kind: typedDict A command argument that is a single delimiter token. | Name | Type | Description | | --- | --- | --- | | `kind` | Literal['Delimiter'] | Discriminator; always ``"Delimiter"``. value: The delimiter value. | | `value` | DelimiterValue | | ## `CSNameArg` Kind: typedDict A command argument that is a control-sequence name. | Name | Type | Description | | --- | --- | --- | | `kind` | Literal['CSName'] | Discriminator; always ``"CSName"``. value: The control-sequence name without the leading backslash. | | `value` | str | | ## `DimensionArg` Kind: typedDict A command argument that is a TeX dimension, such as ``2pt``. | Name | Type | Description | | --- | --- | --- | | `kind` | Literal['Dimension'] | Discriminator; always ``"Dimension"``. value: The dimension as written. | | `value` | str | | ## `IntegerArg` Kind: typedDict A command argument that is an integer literal. | Name | Type | Description | | --- | --- | --- | | `kind` | Literal['Integer'] | Discriminator; always ``"Integer"``. value: The integer as written. | | `value` | str | | ## `KeyValArg` Kind: typedDict A command argument that is a ``key=value`` list. | Name | Type | Description | | --- | --- | --- | | `kind` | Literal['KeyVal'] | Discriminator; always ``"KeyVal"``. value: The raw key-value text. | | `value` | str | | ## `ColumnArg` Kind: typedDict A command argument that is a tabular column specification. | Name | Type | Description | | --- | --- | --- | | `kind` | Literal['Column'] | Discriminator; always ``"Column"``. value: The column specification as written. | | `value` | str | | ## `BooleanArg` Kind: typedDict A command argument that is a star flag, modeled as a boolean. | Name | Type | Description | | --- | --- | --- | | `kind` | Literal['Boolean'] | Discriminator; always ``"Boolean"``. value: ``True`` when the star was present. | | `value` | bool | | ## `ArgRef` Kind: typeAlias ```python MathArg | TextArg | DelimiterArg | OperatorNameArg | CSNameArg | DimensionArg | IntegerArg | KeyValArg | ColumnArg | BooleanArg ``` A read command argument: one of the discriminated argument-value dicts. ## `ArgFormKind` Kind: typeAlias ```python Literal['mandatory', 'optional', 'star', 'group', 'until', 'delimited', 'paired'] ``` ## `NodeSlotKind` Kind: typeAlias ```python Literal['child', 'arg', 'script_base', 'subscript', 'superscript', 'infix_left', 'infix_right', 'env_body'] ``` ## `ArgForm` Kind: typedDict | Name | Type | Description | | --- | --- | --- | | `kind` | ArgFormKind | | | `open` | NotRequired[str] | | | `close` | NotRequired[str] | | ## `OperatorNameArg` Kind: typedDict | Name | Type | Description | | --- | --- | --- | | `form` | ArgForm | | | `kind` | Literal['OperatorName'] | | | `node` | Node | | ## `Paired` Kind: class ```python Paired(value: Node | str, open: str, close: str) -> None ``` Immutable content and delimiter pair for an argspec Paired slot. Pass it as a slot value to select an explicit delimiter pair. ``value`` is a detached node or source written between ``open`` and ``close``. Instances compare and hash by value. ```python doc.create_command("sqrt", [texform.Paired("x", "[", "]"), "y"]) ``` #### `Paired` ```python Paired(value: Node | str, open: str, close: str) -> None ``` | Name | Type | Default | Description | | --- | --- | --- | --- | | `value` | Node | str | — | | | `open` | str | — | | | `close` | str | — | | #### `value` ```python value() -> Node | str ``` **Returns** `Node | str` #### `open` ```python open() -> str ``` **Returns** `str` #### `close` ```python close() -> str ``` **Returns** `str` #### `__eq__` ```python __eq__(other: object) -> bool ``` | Name | Type | Default | Description | | --- | --- | --- | --- | | `other` | object | — | | **Returns** `bool` #### `__hash__` ```python __hash__() -> int ``` **Returns** `int` ## `Arg` Kind: typeAlias ```python Node | str | bool | None | Paired ``` Slot input: source written inside the slot, a detached content node, a star flag, absence, or an explicit ``Paired`` delimiter pair. ## `NodeSlot` Kind: typedDict | Name | Type | Description | | --- | --- | --- | | `kind` | NodeSlotKind | | | `index` | int | None | | --- # TransformEngine Normalize formulas with a profile, tune the pipeline, and read the transform report. https://texform.dev/api/python/transform-engine ## `TransformEngine` Kind: class ```python TransformEngine(profile: TransformProfile, knowledge_base: KnowledgeBase | None = None, disable_rules: list[str] | None = None, default_parse_config: ParseConfig | None = None) -> None ``` Normalize a formula into the canonical form selected by a profile. The engine runs a multi-phase pipeline (LowerAttributes, a fixed-point Rewrite loop, FinalizeAst, FlattenGroups). A profile picks rule levels; ``TransformConfig`` is nested by phase and controls per-run switches without changing the selected rule set. The engine also bundles a parser, so it exposes ``parse`` and shares its immutable ``knowledge_base()``. Since 0.4.0 the bundled parser defaults to ``ParseConfig`` ``LENIENT``, and ``TransformConfig`` is the nested four-phase object (not flat ``rewrite_enabled`` / ``max_iterations`` keys). Per-call ``config`` must be a complete ``TransformConfig`` (or ``ParseConfig`` on ``parse``); a dict passed as ``config`` raises ``ConfigError``. Other keywords are overlays (``normalize(src, rewrite={"enabled": False})``). Nested overlay values must be dicts, not phase-config class instances. Construct a full object with ``TransformConfig(...)`` (all four phases required) or the profile classmethods ``authoring()`` / ``faithful()`` / ``corpus()`` / ``equiv()``. Normalization is gated on a complete tree: an incomplete parse raises ``ParseError``, since normalizing a tree with holes is meaningless. For the conceptual model — profiles, the pipeline, and the eliminated-form contract — see the Transforms guide. ```python import texform engine = texform.TransformEngine(profile="corpus") engine.normalize(r"a \over b") # '\frac { a } { b }' engine.normalize(r"a \over b", rewrite={"enabled": False}) engine.normalize(r"a \over b", reject_unknown=True, flatten_groups={"enabled": False}) reported = engine.normalize_with_report(r"a \over b") reported["normalized"] # '\frac { a } { b }' cfg = engine.default_transform_config() cfg.rewrite.max_iterations = 50 result = engine.parse(r"a \over b") engine.transform(result["document"], cfg) engine.transform_with_report(result["document"], texform.TransformConfig.faithful(), rewrite={"enabled": False}) ``` - [TransformConfig](https://texform.dev/api/python/transform-engine.md#TransformConfig) - [NormalizeReportResult](https://texform.dev/api/python/transform-engine.md#NormalizeReportResult) - [TransformReport](https://texform.dev/api/python/transform-engine.md#TransformReport) - [Parser](https://texform.dev/api/python/parser.md#Parser) - [Transforms](https://texform.dev/docs/transforms.md) #### `TransformEngine` ```python TransformEngine(profile: TransformProfile, knowledge_base: KnowledgeBase | None = None, disable_rules: list[str] | None = None, default_parse_config: ParseConfig | None = None) -> None ``` Construct an engine sharing knowledge; invalid profiles or rules raise ConfigError. | Name | Type | Default | Description | | --- | --- | --- | --- | | `profile` | TransformProfile | — | | | `knowledge_base` | KnowledgeBase | None | None | | | `disable_rules` | list[str] | None | None | | | `default_parse_config` | ParseConfig | None | None | | #### `normalize` ```python normalize(src: str, config: TransformConfig | None = None, **overrides) -> str ``` Parse, transform, and serialize a formula in one call. Normalization is gated on a complete tree. If the input cannot produce a complete tree, ``normalize`` raises ``ParseError`` carrying the diagnostics and the partial document. Empty input is complete and normalizes normally. This path returns the serialized string and does not build a report. | Name | Type | Default | Description | | --- | --- | --- | --- | | `src` | str | — | The LaTeX source string. | | `config` | TransformConfig | None | None | A complete ``TransformConfig``; it replaces only the transform half of the baseline. ``**overrides`` are layered on top. A dict here raises ``ConfigError``. | | `**overrides` | Unpack[NormalizeOverrides] | — | Keyword overlay from ``NormalizeOverrides`` (parse keys plus nested phase dicts). Nested values must be dicts, not config class instances. | **Returns** `str` The serialized LaTeX string. - `ParseError`: If the source does not parse into a complete tree. - `ConfigError`: If ``config`` is not a ``TransformConfig``, a dict is passed as ``config``, or an override key or value is invalid. ```python engine = texform.TransformEngine(profile="corpus") engine.normalize(r"\dv{f}{x}") # '\frac { \mathrm { d } f } { \mathrm { d } x }' ``` - [TransformEngine.normalize_with_report](https://texform.dev/api/python/transform-engine.md#TransformEngine.normalize_with_report) - [TransformEngine.transform](https://texform.dev/api/python/transform-engine.md#TransformEngine.transform) #### `normalize_with_report` ```python normalize_with_report(src: str, config: TransformConfig | None = None, **overrides) -> NormalizeReportResult ``` Parse, transform, and serialize a formula, returning text and a report. Accepts the same source, ``config``, and keyword overlays as ``normalize``. Output text and errors follow that contract. The report is diagnostic: its fields, phase divisions, and counters are not part of the stable compatibility promise. A failure does not return a partial report. | Name | Type | Default | Description | | --- | --- | --- | --- | | `src` | str | — | The LaTeX source string. | | `config` | TransformConfig | None | None | A complete ``TransformConfig``, with the same meaning as on ``normalize``. | | `**overrides` | Unpack[NormalizeOverrides] | — | Keyword overlay from ``NormalizeOverrides``. | **Returns** `NormalizeReportResult` A ``NormalizeReportResult`` dict with ``normalized`` and ``report``. - `ParseError`: If the source does not parse into a complete tree. - `ConfigError`: If ``config`` or an override is invalid. - [TransformEngine.normalize](https://texform.dev/api/python/transform-engine.md#TransformEngine.normalize) - [TransformReport](https://texform.dev/api/python/transform-engine.md#TransformReport) #### `transform` ```python transform(document: Document, config: TransformConfig | None = None, **overrides) -> None ``` Transform a live ``Document`` in place. ``transform`` accepts documents sharing this engine's knowledge-base instance, including constructed, copied, and imported snapshots. The document must also be complete; a document that ``has_errors()`` is rejected with ``TransformError``. This path returns ``None`` and does not build a report. | Name | Type | Default | Description | | --- | --- | --- | --- | | `document` | Document | — | The live ``Document`` to update in place. | | `config` | TransformConfig | None | None | A complete ``TransformConfig``; ``**overrides`` are layered on top. It does not accept parse options. A dict here raises ``ConfigError``. | | `**overrides` | Unpack[TransformOverrides] | — | Keyword overlay from ``TransformOverrides``. Nested values must be dicts, not config class instances. | **Returns** `None` ``None``. - `TransformError`: If the document has a different knowledge base, has parse errors, or on a contract violation. - `ConfigError`: If ``config`` is not a ``TransformConfig``, a dict is passed as ``config``, or an override key or value is invalid. ```python engine = texform.TransformEngine(profile="corpus") result = engine.parse(r"a \over b") document = result["document"] assert document is not None engine.transform(document) document.to_latex() # '\frac { a } { b }' ``` - [TransformEngine.transform_with_report](https://texform.dev/api/python/transform-engine.md#TransformEngine.transform_with_report) - [TransformEngine.normalize](https://texform.dev/api/python/transform-engine.md#TransformEngine.normalize) #### `transform_with_report` ```python transform_with_report(document: Document, config: TransformConfig | None = None, **overrides) -> TransformReport ``` Transform a live ``Document`` in place and return its diagnostic report. Accepts the same document, ``config``, and keyword overlays as ``transform``. Document ownership, completeness, and errors follow that contract. The report is diagnostic and is not part of the stable compatibility promise. A failure does not return a partial report. | Name | Type | Default | Description | | --- | --- | --- | --- | | `document` | Document | — | The live ``Document`` to update in place. | | `config` | TransformConfig | None | None | A complete ``TransformConfig``, with the same meaning as on ``transform``. | | `**overrides` | Unpack[TransformOverrides] | — | Keyword overlay from ``TransformOverrides``. | **Returns** `TransformReport` The diagnostic transform report dict. - `TransformError`: If the document has a different knowledge base, has parse errors, or on a contract violation. - `ConfigError`: If ``config`` or an override is invalid. - [TransformEngine.transform](https://texform.dev/api/python/transform-engine.md#TransformEngine.transform) - [TransformReport](https://texform.dev/api/python/transform-engine.md#TransformReport) #### `parse` ```python parse(src: str, config: ParseConfig | None = None, **overrides) -> ParseResult ``` Parse a LaTeX string using the engine's bundled parser. Uses this engine's parser and default parse config. Engines default to lenient parsing; pass ``config`` to override it per call. A document produced here is owned by this engine and is the only kind ``transform`` accepts. | Name | Type | Default | Description | | --- | --- | --- | --- | | `src` | str | — | The LaTeX source string. | | `config` | ParseConfig | None | None | A complete ``ParseConfig``; ``**overrides`` are layered on top. ``None`` uses the engine default. A dict here raises ``ConfigError``. | | `**overrides` | Unpack[ParseOverrides] | — | Keyword overlay from ``ParseOverrides``. | **Returns** `ParseResult` A ``ParseResult`` dict with ``document`` and ``diagnostics``. - `ConfigError`: If ``config`` is not a ``ParseConfig``, a dict is passed as ``config``, or an override key or value is invalid. - [Parser.parse](https://texform.dev/api/python/parser.md#Parser.parse) - [TransformEngine.transform](https://texform.dev/api/python/transform-engine.md#TransformEngine.transform) #### `default_parse_config` ```python default_parse_config() -> ParseConfig ``` Return a new ``ParseConfig`` holding this engine's parse defaults. **Returns** `ParseConfig` #### `default_transform_config` ```python default_transform_config() -> TransformConfig ``` Return a new ``TransformConfig`` holding this engine's transform defaults. **Returns** `TransformConfig` ### Knowledge-base lookups Query the knowledge the parser loaded. Predicate methods take a single `name`; lookup methods take `name` and a `mode` of `"math"` or `"text"` and return a knowledge-entry dict or `None`. #### `knowledge_base` ```python knowledge_base() -> KnowledgeBase ``` Return the immutable knowledge base shared with parsed documents. **Returns** `KnowledgeBase` ## `TransformConfig` Kind: class ```python TransformConfig(lower_attributes: LowerAttributesConfig, rewrite: RewriteConfig, finalize_ast: FinalizeAstConfig, flatten_groups: FlattenGroupsConfig) -> None ``` Complete transform configuration, nested by pipeline phase. Since 0.4.0 this is the only accepted ``config=`` type for ``transform`` / ``normalize``. A dict passed as ``config`` raises ``ConfigError``; use keyword overlays (``normalize(src, rewrite={"enabled": False})``) or construct this class. The constructor requires all four phase configs; the classmethods ``authoring()`` / ``faithful()`` / ``corpus()`` / ``equiv()`` return the defaults a profile actually uses. The four attributes are shared references: ``cfg.rewrite.enabled = False`` is visible through ``cfg.rewrite.enabled`` and takes effect when the object is passed as ``config``. Nested overlay values must be dicts, not these phase-config instances. ```python config = texform.TransformConfig( lower_attributes=texform.LowerAttributesConfig(), rewrite=texform.RewriteConfig(max_iterations=50), finalize_ast=texform.FinalizeAstConfig(), flatten_groups=texform.FlattenGroupsConfig(enabled=False), ) config.rewrite.max_iterations # 50 texform.TransformConfig.corpus().flatten_groups.preserve_rendered_spacing # False ``` - [TransformEngine](https://texform.dev/api/python/transform-engine.md#TransformEngine) - [RewriteConfig](https://texform.dev/api/python/transform-engine.md#RewriteConfig) - [LowerAttributesConfig](https://texform.dev/api/python/transform-engine.md#LowerAttributesConfig) - [FinalizeAstConfig](https://texform.dev/api/python/transform-engine.md#FinalizeAstConfig) - [FlattenGroupsConfig](https://texform.dev/api/python/transform-engine.md#FlattenGroupsConfig) #### `lower_attributes` The LowerAttributes phase config (shared reference). #### `rewrite` The Rewrite phase config (shared reference). #### `finalize_ast` The FinalizeAst phase config (shared reference). #### `flatten_groups` The FlattenGroups phase config (shared reference). #### `TransformConfig` ```python TransformConfig(lower_attributes: LowerAttributesConfig, rewrite: RewriteConfig, finalize_ast: FinalizeAstConfig, flatten_groups: FlattenGroupsConfig) -> None ``` Construct a transform configuration. All four phases are required. | Name | Type | Default | Description | | --- | --- | --- | --- | | `lower_attributes` | LowerAttributesConfig | — | The LowerAttributes phase config. | | `rewrite` | RewriteConfig | — | The Rewrite phase config. | | `finalize_ast` | FinalizeAstConfig | — | The FinalizeAst phase config. | | `flatten_groups` | FlattenGroupsConfig | — | The FlattenGroups phase config. | - `TypeError`: If a required phase argument is missing. Start with a profile classmethod such as ``TransformConfig.corpus()`` to change only selected fields. #### `authoring` ```python authoring() -> TransformConfig ``` Return the transform config the ``authoring`` profile uses by default. **Returns** `TransformConfig` #### `faithful` ```python faithful() -> TransformConfig ``` Return the transform config the ``faithful`` profile uses by default. **Returns** `TransformConfig` #### `corpus` ```python corpus() -> TransformConfig ``` Return the transform config the ``corpus`` profile uses by default. **Returns** `TransformConfig` #### `equiv` ```python equiv() -> TransformConfig ``` Return the transform config the ``equiv`` profile uses by default. **Returns** `TransformConfig` ## `RewriteConfig` Kind: class ```python RewriteConfig(enabled: bool = True, max_iterations: int = 100) -> None ``` Complete Rewrite phase configuration. Nested under ``TransformConfig.rewrite``. Disabling this phase leaves legacy syntax such as ``\over`` untouched; it does not change which rule levels the profile selected. Overlay with ``rewrite={"enabled": False}``; do not pass this class as a nested overlay value. ```python cfg = texform.RewriteConfig(max_iterations=50) cfg.enabled # True cfg.max_iterations # 50 ``` #### `enabled` Whether the phase runs. Defaults to ``True``. #### `max_iterations` The cap on fixed-point passes. Defaults to ``100``. #### `RewriteConfig` ```python RewriteConfig(enabled: bool = True, max_iterations: int = 100) -> None ``` Construct a Rewrite phase configuration. | Name | Type | Default | Description | | --- | --- | --- | --- | | `enabled` | bool | True | Whether the phase runs. | | `max_iterations` | int | 100 | The cap on fixed-point passes. | ## `LowerAttributesConfig` Kind: class ```python LowerAttributesConfig(enabled: bool = True) -> None ``` Complete LowerAttributes phase configuration. This is a nested field of ``TransformConfig``, not a dict overlay. Pass the parent ``TransformConfig`` as ``config``; to overlay only this phase use ``lower_attributes={"enabled": False}``. A dict passed as ``config`` on ``transform`` / ``normalize`` raises ``ConfigError``. ```python texform.LowerAttributesConfig().enabled # True texform.LowerAttributesConfig(enabled=False).enabled # False ``` #### `enabled` Whether the phase runs. Defaults to ``True``. #### `LowerAttributesConfig` ```python LowerAttributesConfig(enabled: bool = True) -> None ``` Construct a LowerAttributes phase configuration. | Name | Type | Default | Description | | --- | --- | --- | --- | | `enabled` | bool | True | Whether the phase runs. | ## `FinalizeAstConfig` Kind: class ```python FinalizeAstConfig(enabled: bool = True) -> None ``` Complete FinalizeAst phase configuration. Nested under ``TransformConfig.finalize_ast``. The phase performs local AST cleanup such as merging adjacent ``Prime`` nodes and normalizing text sequences. Overlay with ``finalize_ast={"enabled": False}``. ```python texform.FinalizeAstConfig().enabled # True ``` #### `enabled` Whether the phase runs. Defaults to ``True``. #### `FinalizeAstConfig` ```python FinalizeAstConfig(enabled: bool = True) -> None ``` Construct a FinalizeAst phase configuration. | Name | Type | Default | Description | | --- | --- | --- | --- | | `enabled` | bool | True | Whether the phase runs. | ## `FlattenGroupsConfig` Kind: class ```python FlattenGroupsConfig(enabled: bool = True, preserve_rendered_spacing: bool = True) -> None ``` Complete FlattenGroups phase configuration. Nested under ``TransformConfig.flatten_groups``. ``enabled`` governs whether the phase runs. ``preserve_rendered_spacing`` keeps groups whose only public-facing effect is rendered math spacing; it does not control serializer source whitespace (see ``SerializeOptions``). Structural guards stay on even when ``preserve_rendered_spacing`` is ``False``. Constructor defaults match the authoring / faithful strict pair (both ``True``); the ``corpus`` and ``equiv`` profiles leave ``preserve_rendered_spacing`` off. Overlay with ``flatten_groups={"enabled": False}``; a dict passed as ``config`` still raises ``ConfigError``. ```python texform.FlattenGroupsConfig().enabled # True texform.FlattenGroupsConfig().preserve_rendered_spacing # True texform.FlattenGroupsConfig(enabled=False).enabled # False texform.TransformConfig.corpus().flatten_groups.preserve_rendered_spacing # False ``` #### `enabled` Whether the phase runs. Defaults to ``True``. #### `preserve_rendered_spacing` Keep groups whose only public-facing effect is rendered math spacing. Defaults to ``True``. This is not serializer source spacing. #### `FlattenGroupsConfig` ```python FlattenGroupsConfig(enabled: bool = True, preserve_rendered_spacing: bool = True) -> None ``` Construct a FlattenGroups phase configuration. | Name | Type | Default | Description | | --- | --- | --- | --- | | `enabled` | bool | True | Whether the phase runs. | | `preserve_rendered_spacing` | bool | True | Keep groups whose only public-facing effect is rendered math spacing. | ## `NormalizeOverrides` Kind: class Extends: ParseOverrides, TransformOverrides Flat parse+transform overlays for ``normalize``. Combines ``ParseOverrides`` keys with nested phase overlays from ``TransformOverrides``. A dict passed as ``config`` still raises ``ConfigError``; these keys are keywords only (``normalize(src, rewrite={"enabled": False}, reject_unknown=True)``). ```python engine = texform.TransformEngine(profile="corpus") engine.normalize(r"a \over b", reject_unknown=True) # '\frac { a } { b }' ``` ## `TransformOverrides` Kind: typedDict Keyword overlays for ``transform``. Each nested value must be a dict of that phase's overlay keys, not a complete phase-config class instance. | Name | Type | Description | | --- | --- | --- | | `lower_attributes` | LowerAttributesOverrides | Overlay dict for the LowerAttributes phase. | | `rewrite` | RewriteOverrides | Overlay dict for the Rewrite phase. | | `finalize_ast` | FinalizeAstOverrides | Overlay dict for the FinalizeAst phase. | | `flatten_groups` | FlattenGroupsOverrides | None | Overlay dict for the FlattenGroups phase. | ## `RewriteOverrides` Kind: typedDict Keyword overlays for the Rewrite phase. Nested under ``transform`` / ``normalize`` as ``rewrite={...}``. The nested value must be a dict, not a ``RewriteConfig`` instance. | Name | Type | Description | | --- | --- | --- | | `enabled` | bool | Overlay for ``RewriteConfig.enabled``. | | `max_iterations` | int | Overlay for ``RewriteConfig.max_iterations``. | ## `LowerAttributesOverrides` Kind: typedDict Keyword overlays for the LowerAttributes phase. Nested under ``transform`` / ``normalize`` as ``lower_attributes={...}``. The nested value must be a dict, not a ``LowerAttributesConfig`` instance. | Name | Type | Description | | --- | --- | --- | | `enabled` | bool | Overlay for ``LowerAttributesConfig.enabled``. | ## `FinalizeAstOverrides` Kind: typedDict Keyword overlays for the FinalizeAst phase. Nested under ``transform`` / ``normalize`` as ``finalize_ast={...}``. The nested value must be a dict, not a ``FinalizeAstConfig`` instance. | Name | Type | Description | | --- | --- | --- | | `enabled` | bool | Overlay for ``FinalizeAstConfig.enabled``. | ## `FlattenGroupsOverrides` Kind: typedDict Keyword overlays for the FlattenGroups phase. Nested under ``transform`` / ``normalize`` as ``flatten_groups={...}``. The nested value must be a dict, not a ``FlattenGroupsConfig`` instance. | Name | Type | Description | | --- | --- | --- | | `enabled` | bool | None | Overlay for ``FlattenGroupsConfig.enabled``. ``None`` leaves the current value unchanged. | | `preserve_rendered_spacing` | bool | None | Overlay for ``FlattenGroupsConfig.preserve_rendered_spacing``. ``None`` leaves the current value unchanged. | ## `NormalizeReportResult` Kind: typedDict Text plus the diagnostic report from one explicit normalize report call. Plain ``normalize`` returns the string directly and does not build this object. | Name | Type | Description | | --- | --- | --- | | `normalized` | str | The serialized LaTeX string. | | `report` | TransformReport | The diagnostic transform report for this call. | - [TransformReport](https://texform.dev/api/python/transform-engine.md#TransformReport) ## `TransformReport` Kind: typedDict Diagnostic report for one explicit report call. Output text and errors follow the ordinary transform contract. Report fields, phase divisions, and these counters are diagnostic: they are not part of the stable compatibility promise. A disabled phase stays present as zeros or an empty list. The Python report uses snake_case; JavaScript exposes the same data in camelCase. | Name | Type | Description | | --- | --- | --- | | `warnings` | list[TransformWarning] | Unknown input names, deduplicated and sorted by kind and name. | | `lower_attributes` | LowerAttributesReport | LowerAttributes activity, summed across its two passes. | | `rewrite` | RewriteReport | Rewrite iterations and per-rule outcomes. | | `finalize_ast` | FinalizeAstReport | Prime merges and text normalizations. | | `flatten_groups` | FlattenGroupsReport | Flatten actions and guard hits. | - [NormalizeReportResult](https://texform.dev/api/python/transform-engine.md#NormalizeReportResult) - [TransformEngine](https://texform.dev/api/python/transform-engine.md#TransformEngine) ## `TransformWarning` Kind: typedDict An unknown input name whose arguments or scope cannot be guaranteed. | Name | Type | Description | | --- | --- | --- | | `kind` | Literal['unknown-command', 'unknown-environment'] | | | `name` | str | | | `message` | str | | ## `LowerAttributesReport` Kind: typedDict The LowerAttributes phase report. | Name | Type | Description | | --- | --- | --- | | `attributes` | list[AttributeStat] | Per-attribute activity, sorted by ``attr`` then ``value``. | | `eliminated_empty_segments` | int | Empty attribute segments removed by the phase. | ## `AttributeStat` Kind: typedDict Activity for one ``(attr, value)`` pair in LowerAttributes. | Name | Type | Description | | --- | --- | --- | | `attr` | str | Stable attribute name, such as ``"math_font"`` or ``"text_size"``. | | `value` | str | Stable value token for that attribute. | | `consumed` | AttributeFormCounts | Input forms consumed by the phase. | | `redundant` | AttributeFormCounts | Consumed forms that did not change the emitted attribute state. | | `emitted` | AttributeFormCounts | Canonical forms emitted by the phase. | ## `AttributeFormCounts` Kind: typedDict Counts of one attribute form, split by carrier. | Name | Type | Description | | --- | --- | --- | | `declaratives` | int | Count carried by declarative markup such as ``{\bf ...}``. | | `prefixes` | int | Count carried by prefix commands such as ``\mathbf{...}``. | ## `RewriteReport` Kind: typedDict The Rewrite phase report. | Name | Type | Description | | --- | --- | --- | | `iterations` | int | Fixed-point passes, including the final unchanged convergence check. Zero when the phase is disabled or no rules are scheduled. | | `rules` | list[RewriteRuleStat] | Rules that recorded activity, sorted by ``key``. | ## `RewriteRuleStat` Kind: typedDict One rewrite rule's outcome for a single call. | Name | Type | Description | | --- | --- | --- | | `key` | str | Stable rule key. | | `applied_count` | int | Times the rule applied. | | `skipped_count` | int | Times the scheduler matched the rule and ``apply`` returned skipped. | ## `FinalizeAstReport` Kind: typedDict The FinalizeAst phase report. | Name | Type | Description | | --- | --- | --- | | `prime_run_merges` | int | Contiguous ``Prime`` runs merged. Each run counts once. | | `text_normalizations` | int | Text segments or text argument slots whose content changed. | ## `FlattenGroupsReport` Kind: typedDict The FlattenGroups phase report. | Name | Type | Description | | --- | --- | --- | | `actions` | FlattenGroupsActionCounts | Flatten actions that changed the tree. | | `guard_hits` | FlattenGroupsGuardCounts | Preserve-guard hits for this call. | ## `FlattenGroupsActionCounts` Kind: typedDict Flatten actions that changed the tree. | Name | Type | Description | | --- | --- | --- | | `removed_empty` | int | Empty groups removed. | | `replaced_single_child` | int | Groups replaced by their only child. | | `inlined_multi_child` | int | Multi-child groups inlined into the parent. | | `unwrapped_slot` | int | Slot wrappers removed. | ## `FlattenGroupsGuardCounts` Kind: typedDict Preserve-guard hits for one FlattenGroups run. A hit is the first guard branch reached for a group, plus the scripted-base sub-count. ``command_contact_via_scripted_base`` is also included in ``command_contact``. These counters are not a partition of preserved groups. | Name | Type | Description | | --- | --- | --- | | `unknown_command_arguments` | int | Possible arguments following an unknown command. | | `argument_capture` | int | Groups whose removal would let a command before or inside them read the next token as an omitted optional or delimited argument. | | `declarative_scope` | int | Declarative-scope protection. | | `script_base` | int | Script-base protection. | | `env_body` | int | Environment-body protection. | | `infix_scope` | int | Infix-scope protection. | | `command_contact` | int | Command-contact protection. | | `command_contact_via_scripted_base` | int | Command contact established through a scripted base. | | `empty_group` | int | Empty-group protection. | | `lone_atom_spacing_char` | int | Single spacing-character protection. | | `leading_atom_spacing_char` | int | Leading spacing-character protection. | | `delimited_pair` | int | Delimited-pair protection. | ## `TransformProfile` Kind: typeAlias ```python Literal['authoring', 'faithful', 'corpus', 'equiv'] ``` Normalization profile name: ``"authoring"``, ``"faithful"``, ``"corpus"``, or ``"equiv"``. --- # serialize Render a SyntaxNode to LaTeX text, and the output style options. https://texform.dev/api/python/serialize ## `serialize` Kind: function ```python serialize(node: SyntaxNode, **options) -> str ``` Render a ``SyntaxNode`` dict to LaTeX text. This is the free-function counterpart to ``Document.to_latex()``; both take the same keyword options. ``Error`` nodes round-trip their captured snippet, ordinary ``Prime`` nodes emit ``\prime`` symbols, pure prime superscripts serialize compactly as ``f'`` / ``f''``, and the serializer guarantees text idempotency. For the conceptual model, see the Serialization guide. It checks only the structure, not conformance to any knowledge base. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | SyntaxNode | — | A ``SyntaxNode`` dict, typically from ``Document.to_syntax()`` or a stored snapshot. | | `**options` | Unpack[SerializeOptions] | — | Serializer style overlays. Omitted keys and explicit ``None`` keep the default (spaced) style. | **Returns** `str` The serialized LaTeX string. - `ConfigError`: If an option key is unknown or a value has the wrong type. ```python result = texform.Parser().parse(r"x^2") document = result["document"] assert document is not None syntax = document.to_syntax() texform.serialize(syntax) # 'x ^ { 2 }' texform.serialize(syntax, style="compact") # 'x^2' ``` - [SerializeOptions](https://texform.dev/api/python/serialize.md#SerializeOptions) - [Document.to_latex](https://texform.dev/api/python/document.md#Document.to_latex) - [Serialization](https://texform.dev/docs/serialization.md) ## `SerializeOptions` Kind: typedDict Options for ``Document.to_latex``, ``Document.to_tokenized_latex``, and ``serialize``. Passed as keywords. Omitted or ``None`` ``style`` selects the default spaced style. Unknown keys and invalid values raise ``ConfigError``. For a task-oriented walkthrough, see the Serialization guide. | Name | Type | Description | | --- | --- | --- | | `style` | Style | None | ``"spaced"`` separates every token and is byte-stable across releases. ``"compact"`` keeps only required spaces and drops braces around single-token scripts; it preserves parsed structure, but its layout may improve across releases. | ```python doc = texform.Parser().parse(r"\frac{a}{b}+x_i^2")["document"] doc.to_latex() # '\frac { a } { b } + x _ { i } ^ { 2 }' doc.to_latex(style="compact") # '\frac{a}{b}+x_i^2' ``` - [Document.to_latex](https://texform.dev/api/python/document.md#Document.to_latex) - [serialize](https://texform.dev/api/python/serialize.md#serialize) ## `Style` Kind: typeAlias ```python Literal['spaced', 'compact'] ``` Spaced output is byte-stable; compact preserves structure and text idempotency. --- # validate_argspec Validate an argspec string and inspect the parsed argument slots. https://texform.dev/api/python/validate-argspec ## `validate_argspec` Kind: function ```python validate_argspec(spec: str) -> ValidateArgspecResult ``` Validate an xparse-style argspec string and report its parsed slots. Use it to self-check an argspec before injecting a custom command into a ``Parser``. It never raises on a malformed spec; instead the result's ``valid`` is ``False`` and ``error`` carries the reason. | Name | Type | Default | Description | | --- | --- | --- | --- | | `spec` | str | — | The argspec string, such as ``"o m"``, ``"s m{}"``, or ``"d<(,)><[,]>"``. | **Returns** `ValidateArgspecResult` A ``ValidateArgspecResult`` dict: ``valid``, ``error``, ``arg_count``, and ``parsed`` (the per-slot breakdown when valid). ```python texform.validate_argspec("o m") # {'valid': True, 'error': None, 'arg_count': 2, 'parsed': [...]} ``` - [ValidateArgspecResult](https://texform.dev/api/python/validate-argspec.md#ValidateArgspecResult) - [ParsedArgSpecSlot](https://texform.dev/api/python/validate-argspec.md#ParsedArgSpecSlot) ## `ValidateArgspecResult` Kind: typedDict The outcome of validating an argspec string with ``validate_argspec``. On success, ``valid`` is ``True`` and ``arg_count`` / ``parsed`` describe the slots; on failure, ``valid`` is ``False`` and ``error`` carries the reason while ``arg_count`` and ``parsed`` are ``None``. | Name | Type | Description | | --- | --- | --- | | `valid` | bool | Whether the specification string parsed. | | `error` | str | None | The error message when invalid, otherwise ``None``. | | `arg_count` | int | None | The number of argument slots when valid, otherwise ``None``. | | `parsed` | list[ParsedArgSpecSlot] | None | The per-slot breakdown when valid, otherwise ``None``. | - [validate_argspec](https://texform.dev/api/python/validate-argspec.md#validate_argspec) - [ParsedArgSpecSlot](https://texform.dev/api/python/validate-argspec.md#ParsedArgSpecSlot) ## `ParsedArgSpecSlot` Kind: typedDict One parsed argument slot of an xparse-style specification. A presentation-oriented view of the internal argspec model, so tooling can consume argspecs without depending on internal crates. | Name | Type | Description | | --- | --- | --- | | `required` | bool | Whether the argument is mandatory (``True``) or optional. | | `no_leading_space` | bool | Whether leading whitespace before the argument is disallowed. | | `nullable` | bool | Whether an absent optional argument yields a null value. | | `kind` | dict[str, Any] | The value kind the slot accepts; a dict with a ``type`` field, one of the ``ArgSpecKindType`` values. A ``content`` kind also carries a ``mode`` of ``"math"`` or ``"text"``. | | `form` | dict[str, Any] | The syntactic form the slot takes; a dict with a ``type`` field, one of the ``ArgSpecFormType`` values. A ``paired`` form carries a ``pairs`` list of open/close delimiter descriptors. | - [ValidateArgspecResult](https://texform.dev/api/python/validate-argspec.md#ValidateArgspecResult) ## `ArgSpecKindType` Kind: typeAlias ```python Literal['content', 'operatorname', 'delimiter', 'csname', 'dimension', 'integer', 'keyval', 'column', 'star'] ``` Value kind of one parsed argspec slot (content, delimiter, star, ...). ## `ArgSpecFormType` Kind: typeAlias ```python Literal['standard', 'star', 'group', 'until', 'delimited', 'paired'] ``` Surface form of one parsed argspec slot (mandatory, optional group, paired, ...). --- # Utilities List knowledge packages, count formula targets, and the exception classes. https://texform.dev/api/python/utilities The structured library exceptions derive from `TexformError`, so one `except texform.TexformError` catches them: - `TexformError` - `ParseError` — an operation needed a complete tree but parsing produced none. - `EditError` — an invalid `Document` edit. - `ConfigError` — invalid construction or per-call configuration input. - `TransformError` — a transform-engine failure. `Parser.parse` reports malformed LaTeX through its result dict rather than raising; invalid configuration still raises `ConfigError`. Two categories are easy to misattribute. Attaching a node that belongs to another document raises `EditError` (`node belongs to a different document`), rejected before either tree is touched, and passing `TransformEngine.transform` a document that `has_errors()` raises `TransformError` — the precondition belongs to the engine, not to the earlier parse. ## `list_packages` Kind: function ```python list_packages() -> list[PackageInfo] ``` List the built-in knowledge packages with their record counts. The returned names are the package identifiers accepted by the ``packages`` argument of ``KnowledgeBase``. **Returns** `list[PackageInfo]` A list of ``PackageInfo`` dicts, each ``{"name": str, "commands": int, "environments": int}``. ```python texform.list_packages() # [{'name': 'ams', 'commands': 35, 'environments': 28}, ...] ``` - [PackageInfo](https://texform.dev/api/python/utilities.md#PackageInfo) - [Parser](https://texform.dev/api/python/parser.md#Parser) - [TransformEngine](https://texform.dev/api/python/transform-engine.md#TransformEngine) ## `count_targets` Kind: function ```python count_targets(src: str, config: ParseConfig | None = None, knowledge_base: KnowledgeBase | None = None) -> dict[str, int] ``` Count command, environment, and character targets in a LaTeX formula. This is a Python-only helper for corpus analysis. The JavaScript binding does not provide an equivalent function. It parses the source and reports aggregate counts rather than returning a tree, so it requires a complete parse. | Name | Type | Default | Description | | --- | --- | --- | --- | | `src` | str | — | The LaTeX source string. | | `config` | ParseConfig | None | None | A ``ParseConfig``, or ``None`` for the defaults. | | `knowledge_base` | KnowledgeBase | None | None | Shared knowledge, or ``None`` for the process-wide default. | **Returns** `dict[str, int]` A dict mapping a prefixed target key to its occurrence count. Keys are prefixed by kind: ``cmd:`` for commands, ``env:`` for environments, and ``char:`` for named character commands such as ``\alpha``. - `ParseError`: If the source does not parse into a complete tree. ```python texform.count_targets(r"\frac{x}{y} + \alpha") # {'cmd:frac': 1, 'char:alpha': 1} ``` - [list_packages](https://texform.dev/api/python/utilities.md#list_packages) ## `PackageInfo` Kind: typedDict Summary of one built-in knowledge package. | Name | Type | Description | | --- | --- | --- | | `name` | str | The package identifier accepted by the ``packages`` argument of ``KnowledgeBase`` (such as ``"base"`` or ``"ams"``). | | `commands` | int | The number of command records in the package. | | `environments` | int | The number of environment records in the package. | ## `TexformError` Kind: exception Extends: Exception Base class for every exception the library raises. Catching ``TexformError`` catches all of them. Note that ``Parser.parse`` does not raise on malformed input; it reports failures through its result dict instead. - [ParseError](https://texform.dev/api/python/utilities.md#ParseError) - [EditError](https://texform.dev/api/python/utilities.md#EditError) - [ConfigError](https://texform.dev/api/python/utilities.md#ConfigError) - [TransformError](https://texform.dev/api/python/utilities.md#TransformError) ## `ParseError` Kind: exception Extends: TexformError Raised when source could not be parsed into a complete tree. Besides the cases below, ``Document.create_*``, ``parse_fragment``, and the ``set_*`` editing methods raise it when a source string is invalid. ``TransformEngine.normalize`` raises this on input that cannot produce a complete tree, and ``count_targets`` raises it when the source does not parse into a complete tree. The exception then carries the diagnostics and the partial document for inspection. #### `diagnostics` The diagnostics describing the parse failure. #### `document` The partial ``Document``, or ``None`` when no tree was produced. ## `EditError` Kind: exception Extends: TexformError Raised by a ``Document`` editing method on an invalid edit. Triggers include editing a read-only (error) tree, detaching or removing the root node, an out-of-bounds index, a wrong container shape, and attaching a node that belongs to another document. The edit is rejected before it can corrupt the tree. ## `ConfigError` Kind: exception Extends: TexformError Raised on invalid construction or per-call configuration input. Triggers include an unknown knowledge package name or transform profile, a ``config`` argument that is not the matching complete-config class (including a dict passed as ``config``), unknown override keys, wrong scalar types (string or int instead of bool, float instead of int), a list where an object is expected, and a config class instance used as a nested override value. ## `TransformError` Kind: exception Extends: TexformError Raised on a transform-engine failure. Triggers include an eliminated-form contract violation, passing ``TransformEngine.transform`` a document that ``has_errors()``, or one bound to a different knowledge-base instance. ## `ConformanceError` Kind: exception Extends: EditError Raised when a construction, edit, or syntax import violates the knowledge base or the document's structure. #### `path` The tree path where the violation was found. #### `rule` A stable machine-readable rule code. ## `list_rules` Kind: function ```python list_rules() -> list[RuleInfo] ``` List the built-in rewrite rules, sorted by key. The returned keys are the rule identifiers accepted by the ``disable_rules`` argument of ``TransformEngine`` and reported in rewrite statistics. A rule runs only when the engine's profile selects its level and the knowledge base enables one of its packages. **Returns** `list[RuleInfo]` A list of ``RuleInfo`` dicts, each ``{"key": str, "level": str, "fidelity": str, "summary": str, "enabled_by_packages": list[str]}``. ```python [rule for rule in texform.list_rules() if rule["key"] == "base/over-to-frac"] # [{'key': 'base/over-to-frac', 'level': 'authoring', 'fidelity': 'render', ...}] ``` - [RuleInfo](https://texform.dev/api/python/utilities.md#RuleInfo) - [TransformEngine](https://texform.dev/api/python/transform-engine.md#TransformEngine) ## `RuleInfo` Kind: typedDict Summary of one built-in rewrite rule. | Name | Type | Description | | --- | --- | --- | | `key` | str | The stable rule key, such as ``"base/over-to-frac"``, accepted by the ``disable_rules`` argument of ``TransformEngine``. | | `level` | RuleLevel | The first profile that accepts the rule's output. | | `fidelity` | RuleFidelity | The rule's worst-case equivalence guarantee. | | `summary` | str | A one-line description of what the rule does. | | `enabled_by_packages` | list[str] | Packages that make the rule loadable when any one of them is enabled. | ## `RuleFidelity` Kind: typeAlias ```python Literal['math', 'reading', 'render'] ``` A rewrite rule's worst-case equivalence guarantee, least to most faithful: ``"math"`` (meaning preserved, rendering may change), ``"reading"`` (notation content, reading order, and structural roles preserved), or ``"render"`` (rendering equivalent under the reference renderer). ## `RuleLevel` Kind: typeAlias ```python Literal['authoring', 'faithful', 'corpus', 'equiv'] ``` The first profile that accepts a rewrite rule's output as a suitable product. --- # JavaScript API JavaScript conventions, WASM loading, the error hierarchy, and a map of the texform module. https://texform.dev/api/js The `texform` npm package exposes the parser, document model, transform engine, and serializer from one Rust core, compiled to WebAssembly. Install it with `npm install texform`; see [Installation](/docs/installation) for supported runtimes and per-bundler notes. Methods, function names, and option keys use camelCase; the Python binding uses snake_case. Both bindings share the parsing and transform core, but configuration interfaces and some exception types differ. The package ships Node, bundler, and lazy web builds behind one exports map. Importing the bare specifier `texform` resolves to `texform/node` under Node.js and `texform/bundler` under a bundler or browser; import a subpath only to force a build. The Node build loads the WebAssembly module through the filesystem and offers both ESM and CommonJS entry points, and needs no setup. The bundler build initializes WebAssembly during module evaluation, so it runs top-level `await` and your toolchain must treat `.wasm` as an asset. TypeScript declarations are bundled and shared by both builds. Structured data crosses the boundary as plain objects described by interfaces in the bundled declarations, never custom wrapper classes. `Document`, `Node`, `Parser`, and `TransformEngine` are live class instances holding a WASM handle, and each exposes `free()` and `[Symbol.dispose]()` — under a `using` declaration the handle is released at scope exit, otherwise garbage collection reclaims it. A `SyntaxNode` is a plain object snapshot. A `Node`'s `kind` is a lowercase property, unlike Python's capitalized `kind()`; the discriminators inside argument and group views keep their capitalized form (`arg.kind === "Math"`). Configuration is a single overlay layer: `ParseConfig`, `TransformConfig`, and `NormalizeConfig` are plain objects whose keys are all optional, and an omitted key keeps the profile or parser default. Unknown keys, snake_case keys, and wrong scalar types throw `TexformConfigError` with a field path. Enum *values* stay snake_case (`"sub_first"`) even though the keys are camelCase. The parse baseline is lenient. The structured library errors derive from `TexformError` (which extends `Error`) and carry a `kind` discriminator: `TexformParseError`, `TexformEditError`, `TexformConfigError`, and `TexformTransformError`. `Parser.parse` returns malformed-input diagnostics and a `document` that may be `null`; invalid configuration still throws `TexformConfigError`. ## Modules ### [Parser](https://texform.dev/api/js/parser.md) - `Parser` (class): A knowledge-driven LaTeX parser. - `ParserOptions` (interface) - `ParseConfig` (interface): Parser strictness configuration overlay, as a plain object with camelCase keys. All fields are optional; `null` / `undefined` means not set. Unknown keys, snake_case keys, arrays in object positions, and wrong scalar types throw TexformConfigError with a path. - `ParseResult` (interface): The outcome of Parser.parse (and TransformEngine.parse): an optional Document plus the diagnostics emitted while parsing. - `ParseDiagnostic` (interface): One diagnostic emitted while parsing, describing a recovered or rejected condition. - `ParseDiagnosticContext` (interface): One nested context frame attached to a ParseDiagnostic, locating the enclosing construct (such as the command or environment) the error occurred within. - `ParseDiagnosticKind` (typeAlias): The category of a ParseDiagnostic, or `null` when the parser emits a diagnostic without a specific category. - `Span` (interface): A source byte range, half-open as `[start, end)`, indexing into the original LaTeX input. - `SyntaxNode` (typeAlias): The lossless, immutable parse snapshot — the single serde wire format. - `ContextItem` (typeAlias): One custom knowledge entry supplied to KnowledgeBase through the `items` option, discriminated by `target`. - `CommandInfo` (interface): The knowledge-base entry for a command, returned by KnowledgeBase.lookupCommand and related lookups. - `EnvInfo` (interface): The knowledge-base entry for an environment, returned by KnowledgeBase.lookupEnv. - `CharacterInfo` (interface): The knowledge-base entry for a special character, returned by KnowledgeBase.lookupCharacter. - `CharacterAttributesInfo` (interface): Rendering attributes for a special character in the knowledge base. - `AllowedMode` (typeAlias): The content modes a command, environment, or character is allowed in: `"math"`, `"text"`, or `"both"`. - `CommandKind` (typeAlias): The syntactic role of a command in the knowledge base: `"prefix"` (takes its arguments after the control sequence), `"infix"` (binds operands on both sides, like `\over`), or `"declarative"` (a scope-affecting declaration). - `ArgSpecKindInfo` (typeAlias): The value kind one parsed argument slot accepts, with a `type` discriminator. - `DelimiterTokenInfo` (typeAlias): A single delimiter token in an argspec form: a literal `char` (such as `(`) or a `control-seq` (such as `langle`). - `ArgSpecFormInfo` (typeAlias): The syntactic form one parsed argument slot takes, with a `type` discriminator: a `standard` argument, a `star` flag, a brace `group`, a `delimited` argument bounded by an `open`/`close` pair, or a `paired` argument accepting any of several interchangeable delimiter `pairs`. ### [KnowledgeBase](https://texform.dev/api/js/knowledge-base.md) - `KnowledgeBase` (class): Immutable knowledge shared by parsers, engines and documents. - `KnowledgeBaseOptions` (interface): Construction options for an immutable KnowledgeBase. - `DelimiterInfo` (interface) - `ConstructionOptions` (interface): Options for the `create*` methods. `mode` is the context mode of the detached subtree. It defaults to text for `createInlineMath`, math for `createDelimitedGroup`, `createScripted`, `createPrime`, and `createInfix`, the group's own mode for `createGroup`, and the document root mode otherwise. ### [Document](https://texform.dev/api/js/document.md) - `Document` (class): The editable LaTeX document tree — the working format you read, mutate, serialize, and transform. - `NodeSpanEntry` (interface): One entry of Document.nodeSpans: a node identifier paired with its recorded source Span. - `TokenizedLatex` (interface): Serialized LaTeX and tokens produced by the same serializer traversal. - `SerializationToken` (interface): One non-empty serialization fragment. - `SerializationTokenKind` (typeAlias): Closed semantic categories emitted by tokenized serialization. - `NodeTable` (interface): Preorder node columns; field names stay snake_case in both bindings. - `ArgumentTable` (interface): All argument slots, ordered by owner row and slot index. - `ColumnarTree` (interface): Tree columns for bulk structural analysis with Arrow or DataFrame tools. ### [Node](https://texform.dev/api/js/node.md) - `Node` (class): A read-only handle into a Document. - `NodeKind` (typeAlias): The runtime kind of a Node, read from the lowercase Node.kind property. - `SyntaxContentMode` (typeAlias): Content mode as it appears in a SyntaxNode snapshot: `"Math"` or `"Text"` (capitalized snapshot form). - `RuntimeContentMode` (typeAlias): Content mode as reported by the runtime tree API: `"math"` or `"text"` (lowercase form used by Node.contentMode and the `mode` arguments of lookup methods). - `GroupKind` (typeAlias): The kind of a group in SyntaxNode snapshot form: an `Explicit` brace group, an `Implicit` group, a `Delimited` group carrying its `left`/`right` delimiters, or an `InlineMath` group. - `GroupKindRef` (typeAlias): The kind of a group in runtime view form, with a `kind` discriminator. Returned by Node.groupKind; the discriminator keeps its capitalized form (`'Explicit'`, `'Delimited'`, ...). A `Delimited` group carries `left` and `right` delimiter strings. - `Delimiter` (typeAlias): A delimiter token in SyntaxNode snapshot form: the literal `"None"`, a single character (`{ Char }`), or a control sequence (`{ Control }`). Runtime views use delimiter strings instead: `.`, one character, or a backslash-prefixed control sequence such as `\langle`. - `Argument` (interface): A present command/environment argument in SyntaxNode snapshot form, pairing its syntactic ArgumentKind with its parsed ArgumentValue. - `ArgumentKind` (typeAlias): The syntactic kind of a command argument in SyntaxNode snapshot form (mandatory, optional, starred, group, until, or delimited/paired). Until forms carry `close`; delimited/paired forms carry `open` and `close`. - `ArgumentValue` (typeAlias): The parsed value carried by an Argument in snapshot form: math or text content, a delimiter, a control-sequence name, a dimension, an integer, a key=value list, a column spec, or a boolean flag. - `ArgumentSlot` (typeAlias): An argument slot of a Node's argument list: either a present Argument or an empty slot (`null`/`undefined`). - `ArgRef` (typeAlias): A command/environment argument in runtime view form, returned by Node.arg and Node.argSlots. - `ArgForm` (typeAlias): The delimiter form of an argument slot; `close` is present for `until`, and both `open` and `close` for `delimited` and `paired` forms. - `NodeSlot` (typeAlias): The parent slot a Node fills; `index` is set for indexed slots such as children and arguments. - `Arg` (typeAlias): A slot value accepted by construction and editing methods: a `string` is the source written inside the slot's boundaries, a detached Node is content, `boolean` sets a star slot, `null` omits an optional slot, and `{ value, open, close }` selects an explicit delimiter pair. ### [TransformEngine](https://texform.dev/api/js/transform-engine.md) - `TransformEngine` (class): A profile-based normalization engine: a parser paired with a transform pipeline. - `TransformEngineOptions` (interface): Construction options for TransformEngine: a shared knowledge base, the required normalization Profile, and optional rule disabling. - `NormalizeConfig` (interface): Per-run overlay for TransformEngine.normalize: parse keys from ParseConfig plus transform keys from TransformConfig, flattened together. `null` / `undefined` / omitted means not set. - `TransformConfig` (interface): The nested, per-phase transform configuration, overriding a profile's transform defaults. - `RewriteConfig` (interface): Per-run switches for the fixed-point Rewrite phase. - `LowerAttributesConfig` (interface): Per-run switches for the LowerAttributes phase. - `FinalizeAstConfig` (interface): Per-run switches for the FinalizeAst phase. - `FlattenGroupsConfig` (interface): Per-run switches for the FlattenGroups phase (redundant-brace removal). - `NormalizeReportResult` (interface): Text plus the diagnostic report from TransformEngine.normalizeWithReport. - `TransformReport` (interface): Diagnostic report for one explicit report call. - `TransformWarning` (interface): An unknown input name whose arguments or scope cannot be guaranteed. - `LowerAttributesReport` (interface): The LowerAttributes phase report. - `AttributeStat` (interface): Activity for one `(attr, value)` pair in LowerAttributes. - `AttributeFormCounts` (interface): Counts of an attribute form split by carrier: `declaratives` (scope-affecting declarations such as `{\bf ...}`) and `prefixes` (prefix commands such as `\mathbf{...}`). - `RewriteRuleStat` (interface): One rewrite rule's outcome for a single call. - `RewriteReport` (interface): The Rewrite phase report. - `FinalizeAstReport` (interface): The FinalizeAst phase report. - `FlattenGroupsActionCounts` (interface): Flatten actions that changed the tree. - `FlattenGroupsGuardCounts` (interface): Preserve-guard hits for one FlattenGroups run. - `FlattenGroupsReport` (interface): The FlattenGroups phase report. - `TransformProfile` (typeAlias): Normalization profile selecting the canonical form for one downstream scenario: `authoring` (polished author-facing output), `faithful` (same rendered formula, convenience macros expanded), `corpus` (training-data normalization, layout hints dropped), or `equiv` (aggressive canonicalization for equivalence comparison). - `Profile` (typeAlias): Normalization profile passed to TransformEngineOptions: `"authoring"`, `"faithful"`, `"corpus"`, or `"equiv"`. - `RuleLevel` (typeAlias): How aggressively a transform rule may rewrite, layered from least to most destructive of stylistic detail. ### [serialize](https://texform.dev/api/js/serialize.md) - `serialize` (function): **Deprecated.** Prefer `Document.fromSyntax(node).toLatex(options)` or `document.toLatex(options)`. This remains exported as a compatibility helper for older snapshot call sites and takes the same options as `toLatex`. - `SerializeOptions` (interface): Options for Document.toLatex, Document.toTokenizedLatex, and serialize. Unknown keys and invalid values throw `TexformConfigError`. For a task-oriented walkthrough, see the Serialization guide. - `Style` (typeAlias): Spaced output is byte-stable; compact preserves structure and text idempotency. ### [validateArgspec](https://texform.dev/api/js/validate-argspec.md) - `validateArgspec` (function): Validate an xparse-style argspec string and report its parsed slots. - `ValidateArgspecResult` (interface): The outcome of validateArgspec: whether the spec is well-formed and, when valid, its parsed slots. - `ParsedArgSpecSlot` (interface): One parsed argument slot of an argspec, as reported by validateArgspec and by the parser's command/environment lookups. ### [Utilities](https://texform.dev/api/js/utilities.md) - `listPackages` (function): List all built-in knowledge packages with record counts. - `PackageInfo` (interface): Summary of one built-in knowledge package, returned by listPackages. - `TexformError` (exception): Base class for every error the library throws. - `TexformParseError` (exception): Thrown when source could not be parsed into a complete tree. - `TexformEditError` (exception): Thrown by Document editing methods on an invalid edit: editing a read-only (error) tree, detaching or removing the root, an out-of-bounds index, or mixing nodes across documents. - `TexformConfigError` (exception): Thrown on invalid construction or per-call configuration: unknown package or profile names, unknown or snake_case keys, arrays in object positions, and wrong scalar types. Messages include a camelCase field path. - `TexformTransformError` (exception): Thrown on a transform-engine failure, such as an eliminated-form contract violation, or passing TransformEngine.transform a document that Document.hasErrors or that is bound to a different KnowledgeBase instance. - `TexformConformanceError` (exception): Thrown when a construction, edit, or syntax import violates the knowledge base or the document's structure. - `listRules` (function): List all built-in rewrite rules, sorted by key. - `RuleInfo` (interface): Summary of one built-in rewrite rule, returned by listRules. - `RuleFidelity` (typeAlias): A rewrite rule's worst-case equivalence guarantee over its declared input domain, ordered least to most faithful: `math` (mathematical meaning is preserved, rendering may change), `reading` (notation content, reading order, and structural roles are preserved), `render` (rendering is equivalent under the reference renderer). --- # Parser Construct a Parser, parse LaTeX, configure strictness, and inspect the knowledge base. https://texform.dev/api/js/parser ## `Parser` Kind: class ```ts Parser(options: ParserOptions | null | undefined) ``` A knowledge-driven LaTeX parser. `Parser` turns LaTeX source into a [ParseResult](/api/js/parser#ParseResult) by consulting the knowledge base for command and environment signatures. It does not normalize — use [TransformEngine](/api/js/transform-engine#TransformEngine) for that. Parsing never throws on malformed input; failures surface in the result. For the conceptual model, see the Parsing guide. Since 0.4.0 the default is [ParseConfig](/api/js/parser#ParseConfig) `LENIENT` (`rejectUnknown: false`, `abortOnError: false`). Per-call `config` is a camelCase overlay object: `null` / `undefined` / omitted means not set. Unknown keys, snake_case keys, arrays in object positions, and wrong scalar types throw [TexformConfigError](/api/js/utilities#TexformConfigError) with a camelCase path. Omit `knowledgeBase` to share the default instance, or construct a [KnowledgeBase](/api/js/knowledge-base#KnowledgeBase) to customize packages and entries. ```ts import { Parser, KnowledgeBase } from 'texform'; const parser = new Parser(); parser.parse(String.raw`\frac{x}{y}`).document?.toLatex(); // '\\frac { x } { y }' const restricted = new Parser({ knowledgeBase: new KnowledgeBase({ packages: ['base', 'ams'] }) }); ``` - [ParseResult](https://texform.dev/api/js/parser.md#ParseResult) - [TransformEngine](https://texform.dev/api/js/transform-engine.md#TransformEngine) - [ParseConfig](https://texform.dev/api/js/parser.md#ParseConfig) - [TexformConfigError](https://texform.dev/api/js/utilities.md#TexformConfigError) - [KnowledgeBase](https://texform.dev/api/js/knowledge-base.md#KnowledgeBase) - [Parsing](https://texform.dev/docs/parsing.md) #### `Parser` ```ts Parser(options: ParserOptions | null | undefined) ``` Construct a parser with an optional shared knowledge base. | Name | Type | Default | Description | | --- | --- | --- | --- | | `options` | ParserOptions | null | undefined | — | - A [ParserOptions](/api/js/parser#ParserOptions) object, or omit/`null` to load the default runtime packages with no customization. | #### `parse` ```ts parse(src: string, config: ParseConfig | null | undefined): ParseResult ``` Parse a LaTeX string into a [ParseResult](/api/js/parser#ParseResult). Never throws on malformed input: the result has one of three honest states (no tree, complete tree, or partial tree with `Error` nodes). Empty input (`''`) yields a clean, complete document, not `null`. | Name | Type | Default | Description | | --- | --- | --- | --- | | `src` | string | — | - The LaTeX source string. | | `config` | ParseConfig | null | undefined | — | - A [ParseConfig](/api/js/parser#ParseConfig) overlay, or omit/`null` for the defaults. | **Returns** `ParseResult` The parse result (`document` and `diagnostics`). ```ts const result = new Parser().parse(String.raw`\frac{x}{y}`); const document = result.document; const diagnostics = result.diagnostics; ``` - [ParseResult](https://texform.dev/api/js/parser.md#ParseResult) - [ParseConfig](https://texform.dev/api/js/parser.md#ParseConfig) #### `defaultParseConfig` ```ts defaultParseConfig(): Complete ``` Return this parser's complete default [ParseConfig](/api/js/parser#ParseConfig). Every field is present. Use it as a starting point when you want to change one key and pass the rest through. **Returns** `Complete` - [ParseConfig](https://texform.dev/api/js/parser.md#ParseConfig) ### Knowledge-base lookups Query the knowledge the parser loaded. Predicate methods take a single `name`; lookup methods take `name` and a `mode` of `"math"` or `"text"` and return a knowledge-entry object or `null`. #### `knowledgeBase` ```ts knowledgeBase(): KnowledgeBase ``` **Returns** `KnowledgeBase` ### Lifecycle #### `free` ```ts free(): void ``` Release the WASM handle for this parser immediately. Prefer a `using` declaration so `[Symbol.dispose]()` runs at scope exit. After `free()`, further use is invalid. Skipping `free()` is safe: garbage collection still reclaims the handle, just not at a determined time. Call `free()` (or use `using`) when many short-lived parsers would otherwise accumulate WASM memory. **Returns** `void` #### `[Symbol.dispose]` ```ts [Symbol.dispose](): void ``` Release the WASM handle. Same as [Parser.free](/api/js/parser#Parser.free); used by `using`. **Returns** `void` - [Parser.free](https://texform.dev/api/js/parser.md#Parser.free) ## `ParserOptions` Kind: interface | Name | Type | Description | | --- | --- | --- | | `knowledgeBase` | KnowledgeBase | undefined | | | `defaultParseConfig` | ParseConfig | null | undefined | Overlay applied to [ParseConfig](/api/js/parser#ParseConfig) `LENIENT` as this parser's default. | ## `ParseConfig` Kind: interface Parser strictness configuration overlay, as a plain object with camelCase keys. All fields are optional; `null` / `undefined` means not set. Unknown keys, snake_case keys, arrays in object positions, and wrong scalar types throw [TexformConfigError](/api/js/utilities#TexformConfigError) with a path. The two boolean axes are orthogonal: [ParseConfig.rejectUnknown](/api/js/parser#ParseConfig.rejectUnknown) controls how unknown names are handled, while [ParseConfig.abortOnError](/api/js/parser#ParseConfig.abortOnError) controls whether the parser stops at the first error or keeps recovering. Neither implies the other, and neither is equivalent to the resulting tree's [Document.hasErrors](/api/js/document#Document.hasErrors) signal. | Name | Type | Description | | --- | --- | --- | | `rejectUnknown` | boolean | undefined | Reject unknown command and environment names. Default `false`. When `true`, an unknown command or environment becomes an `unknown-command` / `unknown-environment` diagnostic. When `false`, unknown names are preserved as `known: false` nodes for lenient exploration. | | `abortOnError` | boolean | undefined | Stop at the first error per item instead of recovering. Default `false`. When `true` (strict), the parser stops at the first error per item and produces no recovery `Error` nodes. When `false` (lenient), it keeps collecting diagnostics and emits `Error` placeholder nodes. The max-group-depth guard emits an `Error` node regardless of this setting. This is a parse-strictness knob, distinct from [Document.hasErrors](/api/js/document#Document.hasErrors), which is a property of the resulting tree. | | `maxGroupDepth` | number | undefined | Maximum group nesting depth before the parser aborts a group. Default `128`. Exceeding it emits a `max-group-depth-exceeded` diagnostic and an `Error` node unconditionally. | ```ts new Parser().parse(String.raw`\foobar{x}`, { rejectUnknown: true, abortOnError: true, }); ``` - [TexformConfigError](https://texform.dev/api/js/utilities.md#TexformConfigError) - [ParseConfig.rejectUnknown](https://texform.dev/api/js/parser.md#ParseConfig.rejectUnknown) - [ParseConfig.abortOnError](https://texform.dev/api/js/parser.md#ParseConfig.abortOnError) - [Document.hasErrors](https://texform.dev/api/js/document.md#Document.hasErrors) ## `ParseResult` Kind: interface The outcome of [Parser.parse](/api/js/parser#Parser.parse) (and [TransformEngine.parse](/api/js/transform-engine#TransformEngine.parse)): an optional [Document](/api/js/document#Document) plus the diagnostics emitted while parsing. There are exactly three honest states, and `parse` never throws on malformed input nor fabricates a placeholder tree: - **Hard failure** — `document` is `null`; the failure is described entirely by `diagnostics`. - **Clean parse** — `document` is a complete, editable tree with `hasErrors() === false`. Empty input (`''`) lands here, not in `null`. - **Partial parse** — `document` is present with `hasErrors() === true`; recovery preserved the unparseable parts as read-only `Error` nodes. | Name | Type | Description | | --- | --- | --- | | `document` | Document | null | The parsed document, or `null` on a hard failure with no tree. | | `diagnostics` | ParseDiagnostic[] | Diagnostics emitted while parsing; may be empty on a clean parse. | - [Parser.parse](https://texform.dev/api/js/parser.md#Parser.parse) - [TransformEngine.parse](https://texform.dev/api/js/transform-engine.md#TransformEngine.parse) - [Document](https://texform.dev/api/js/document.md#Document) ## `ParseDiagnostic` Kind: interface One diagnostic emitted while parsing, describing a recovered or rejected condition. Diagnostics are a separate channel from `Error` placeholder nodes: a diagnostic does not by itself make an otherwise editable tree read-only. Within a parser-produced result, however, a tree that [Document.hasErrors](/api/js/document#Document.hasErrors) always comes with at least one diagnostic. | Name | Type | Description | | --- | --- | --- | | `kind` | ParseDiagnosticKind | null | The diagnostic category, or `null` when uncategorized. | | `message` | string | Human-readable description of the problem. | | `span` | Span | Source span the diagnostic points at. | | `expected` | string[] | Tokens the parser expected at this position; may be empty. | | `found` | string | null | The token actually found, or `null`. | | `contexts` | ParseDiagnosticContext[] | Enclosing-construct frames, outermost-relevant first; may be empty. | - [Document.hasErrors](https://texform.dev/api/js/document.md#Document.hasErrors) - [ParseResult](https://texform.dev/api/js/parser.md#ParseResult) ## `ParseDiagnosticContext` Kind: interface One nested context frame attached to a [ParseDiagnostic](/api/js/parser#ParseDiagnostic), locating the enclosing construct (such as the command or environment) the error occurred within. | Name | Type | Description | | --- | --- | --- | | `label` | string | Human-readable label for the enclosing construct. | | `span` | Span | Source span of the enclosing construct. | - [ParseDiagnostic](https://texform.dev/api/js/parser.md#ParseDiagnostic) ## `ParseDiagnosticKind` Kind: typeAlias ```ts "ambiguous-infix" | "argument-validation" | "command-mode-error" | "comment-truncated-argument" | "environment-mode-error" | "environment-name-mismatch" | "invalid-character" | "left-right-delimiter" | "max-group-depth-exceeded" | "raw-expected-found" | "text-script-error" | "unclosed-environment" | "unclosed-inline-math" | "unexpected-math-shift" | "unknown-command" | "unknown-environment" ``` The category of a [ParseDiagnostic](/api/js/parser#ParseDiagnostic), or `null` when the parser emits a diagnostic without a specific category. Each variant names a distinct recoverable or rejected parse condition, such as an unknown command, an argument-validation failure, or an unclosed inline math span. - [ParseDiagnostic](https://texform.dev/api/js/parser.md#ParseDiagnostic) ## `Span` Kind: interface A source byte range, half-open as `[start, end)`, indexing into the original LaTeX input. | Name | Type | Description | | --- | --- | --- | | `start` | number | Inclusive start byte offset. | | `end` | number | Exclusive end byte offset. | ## `SyntaxNode` Kind: typeAlias ```ts { Root: { mode: SyntaxContentMode; children: SyntaxNode[] } } | { Group: { mode: SyntaxContentMode; kind: GroupKind; children: SyntaxNode[]; }; } | { Command: { name: string; args: ArgumentSlot[]; known: boolean } } | { Infix: { name: string; args: ArgumentSlot[]; left: SyntaxNode; right: SyntaxNode; }; } | { Declarative: { name: string; args: ArgumentSlot[] } } | { Environment: { name: string; args: ArgumentSlot[]; known: boolean; body: SyntaxNode; }; } | { Scripted: { base: SyntaxNode; subscript?: SyntaxNode; superscript?: SyntaxNode; }; } | { Prime: { count: number } } | { Error: { message: string; snippet: string } } | { Text: string } | { Char: string } | "ActiveSpace" | "AlignmentTab" ``` The lossless, immutable parse snapshot — the single serde wire format. `SyntaxNode` is a plain object (not a class instance) that backs JSON snapshots, transport across the binding, and test fixtures. It can represent a partial parse: unparseable fragments survive as `Error` nodes. It carries no editing behavior; bridge it into an editable tree with [Document.fromSyntax](/api/js/document#Document.fromSyntax), and recover one with [Document.toSyntax](/api/js/document#Document.toSyntax). The two conversions are symmetric over every node kind, including `Error` and `Prime`. A `Prime` node represents symbols in the current math list; `Scripted` supplies any superscript binding. A leading quote has an empty group as its scripted base. External snapshots cannot infer the old intent of a bare `Prime` node; reparse the source or update the tree explicitly. - [Document.fromSyntax](https://texform.dev/api/js/document.md#Document.fromSyntax) - [Document.toSyntax](https://texform.dev/api/js/document.md#Document.toSyntax) - [Document](https://texform.dev/api/js/document.md#Document) ## `ContextItem` Kind: typeAlias ```ts { target: "command"; name: string; kind: CommandKind; allowedMode: string; argspec: string; tags?: string[]; } | { target: "environment"; name: string; allowedMode: string; bodyMode: RuntimeContentMode; argspec: string; tags?: string[]; } | { target: "delimiter"; name: string; } ``` One custom knowledge entry supplied to [KnowledgeBase](/api/js/knowledge-base#KnowledgeBase) through the `items` option, discriminated by `target`. A `command` entry carries its kind, allowed mode, and `argspec`; an `environment` entry additionally carries its `bodyMode`; a `delimiter` entry registers a delimiter-control name. Unknown keys and disallowed fields throw [TexformConfigError](/api/js/utilities#TexformConfigError). - [KnowledgeBase](https://texform.dev/api/js/knowledge-base.md#KnowledgeBase) - [TexformConfigError](https://texform.dev/api/js/utilities.md#TexformConfigError) ## `CommandInfo` Kind: interface The knowledge-base entry for a command, returned by [KnowledgeBase.lookupCommand](/api/js/knowledge-base#KnowledgeBase.lookupCommand) and related lookups. | Name | Type | Description | | --- | --- | --- | | `name` | string | The command name, without the leading backslash. | | `kind` | CommandKind | The command's syntactic role (prefix, infix, declarative). | | `allowedMode` | string | The modes the command is allowed in. | | `specString` | string | The raw xparse-style argument-specification string. | | `fromPackages` | string[] | The knowledge packages that define this command. | | `tags` | string[] | Free-form classification tags. | | `args` | ParsedArgSpecSlot[] | The parsed argument slots, mirroring [validateArgspec](/api/js/validate-argspec#validateArgspec) output. | - [KnowledgeBase.lookupCommand](https://texform.dev/api/js/knowledge-base.md#KnowledgeBase.lookupCommand) ## `EnvInfo` Kind: interface The knowledge-base entry for an environment, returned by [KnowledgeBase.lookupEnv](/api/js/knowledge-base#KnowledgeBase.lookupEnv). | Name | Type | Description | | --- | --- | --- | | `name` | string | The environment name. | | `allowedMode` | string | The modes the environment is allowed in. | | `bodyMode` | RuntimeContentMode | The content mode the environment body is parsed in. | | `specString` | string | The raw xparse-style argument-specification string. | | `fromPackages` | string[] | The knowledge packages that define this environment. | | `tags` | string[] | Free-form classification tags. | | `args` | ParsedArgSpecSlot[] | The parsed argument slots for the environment's arguments. | - [KnowledgeBase.lookupEnv](https://texform.dev/api/js/knowledge-base.md#KnowledgeBase.lookupEnv) ## `CharacterInfo` Kind: interface The knowledge-base entry for a special character, returned by [KnowledgeBase.lookupCharacter](/api/js/knowledge-base#KnowledgeBase.lookupCharacter). | Name | Type | Description | | --- | --- | --- | | `name` | string | The character name, without the leading backslash. | | `allowedMode` | string | The modes the character is allowed in. | | `unicodeValue` | string | The Unicode code point the character maps to, as a string. | | `attributes` | CharacterAttributesInfo | Rendering attributes such as `mathvariant`. | | `package` | string | The knowledge package that defines this character. | - [KnowledgeBase.lookupCharacter](https://texform.dev/api/js/knowledge-base.md#KnowledgeBase.lookupCharacter) ## `CharacterAttributesInfo` Kind: interface Rendering attributes for a special character in the knowledge base. | Name | Type | Description | | --- | --- | --- | | `mathvariant` | string | null | The MathML `mathvariant` the character renders with, or `null`. | ## `AllowedMode` Kind: typeAlias ```ts "math" | "text" | "both" ``` The content modes a command, environment, or character is allowed in: `"math"`, `"text"`, or `"both"`. ## `CommandKind` Kind: typeAlias ```ts "prefix" | "infix" | "declarative" ``` The syntactic role of a command in the knowledge base: `"prefix"` (takes its arguments after the control sequence), `"infix"` (binds operands on both sides, like `\over`), or `"declarative"` (a scope-affecting declaration). ## `ArgSpecKindInfo` Kind: typeAlias ```ts { type: "content"; mode: RuntimeContentMode } | { type: "operatorname" } | { type: "delimiter" } | { type: "csname" } | { type: "dimension" } | { type: "integer" } | { type: "keyval" } | { type: "column" } | { type: "star" } ``` The value kind one parsed argument slot accepts, with a `type` discriminator. `content` carries the [RuntimeContentMode](/api/js/node#RuntimeContentMode) its body parses in; the other variants are leaf kinds (operatorname, delimiter, csname, dimension, integer, keyval, column, star). - [RuntimeContentMode](https://texform.dev/api/js/node.md#RuntimeContentMode) - [ParsedArgSpecSlot](https://texform.dev/api/js/validate-argspec.md#ParsedArgSpecSlot) ## `DelimiterTokenInfo` Kind: typeAlias ```ts { type: "char"; value: string } | { type: "control-seq"; value: string } ``` A single delimiter token in an argspec form: a literal `char` (such as `(`) or a `control-seq` (such as `langle`). ## `ArgSpecFormInfo` Kind: typeAlias ```ts { type: "standard" } | { type: "star" } | { type: "group" } | { type: "until"; close: DelimiterTokenInfo } | { type: "delimited"; open: DelimiterTokenInfo; close: DelimiterTokenInfo } | { type: "paired"; pairs: Array<{ open: DelimiterTokenInfo; close: DelimiterTokenInfo }>; } ``` The syntactic form one parsed argument slot takes, with a `type` discriminator: a `standard` argument, a `star` flag, a brace `group`, a `delimited` argument bounded by an `open`/`close` pair, or a `paired` argument accepting any of several interchangeable delimiter `pairs`. - [ParsedArgSpecSlot](https://texform.dev/api/js/validate-argspec.md#ParsedArgSpecSlot) --- # KnowledgeBase Share immutable knowledge across parsers, transform engines, and documents. https://texform.dev/api/js/knowledge-base ## `KnowledgeBase` Kind: class ```ts KnowledgeBase(options: KnowledgeBaseOptions | undefined) ``` Immutable knowledge shared by parsers, engines and documents. It defines the known commands, environments, characters, and delimiters that parsing, transforms, and `Document` construction and edits check against. Share one instance across a [Parser](/api/js/parser#Parser), [TransformEngine](/api/js/transform-engine#TransformEngine), and their documents. Enumerations are sorted by name and returned as copies. ```ts const knowledgeBase = new KnowledgeBase({ packages: ['ams'] }); const { document } = new Parser({ knowledgeBase }).parse(String.raw`\frac{a}{b}`); ``` - [Parser](https://texform.dev/api/js/parser.md#Parser) - [TransformEngine](https://texform.dev/api/js/transform-engine.md#TransformEngine) #### `KnowledgeBase` ```ts KnowledgeBase(options: KnowledgeBaseOptions | undefined) ``` Build the knowledge base once. Throws [TexformConfigError](/api/js/utilities#TexformConfigError) for an invalid package name or item. | Name | Type | Default | Description | | --- | --- | --- | --- | | `options` | KnowledgeBaseOptions | undefined | — | | #### `free` ```ts free(): void ``` **Returns** `void` #### `[Symbol.dispose]` ```ts [Symbol.dispose](): void ``` **Returns** `void` #### `isSame` ```ts isSame(other: KnowledgeBase): boolean ``` Whether `other` is the same instance; equality is identity, not recipe. | Name | Type | Default | Description | | --- | --- | --- | --- | | `other` | KnowledgeBase | — | | **Returns** `boolean` #### `packages` ```ts packages(): string[] ``` **Returns** `string[]` #### `commands` ```ts commands(mode: RuntimeContentMode): CommandInfo[] ``` | Name | Type | Default | Description | | --- | --- | --- | --- | | `mode` | RuntimeContentMode | — | | **Returns** `CommandInfo[]` #### `environments` ```ts environments(mode: RuntimeContentMode): EnvInfo[] ``` | Name | Type | Default | Description | | --- | --- | --- | --- | | `mode` | RuntimeContentMode | — | | **Returns** `EnvInfo[]` #### `characters` ```ts characters(mode: RuntimeContentMode): CharacterInfo[] ``` | Name | Type | Default | Description | | --- | --- | --- | --- | | `mode` | RuntimeContentMode | — | | **Returns** `CharacterInfo[]` #### `delimiters` ```ts delimiters(): DelimiterInfo[] ``` **Returns** `DelimiterInfo[]` #### `isDelimiterControl` ```ts isDelimiterControl(name: string): boolean ``` | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | string | — | | **Returns** `boolean` #### `knowsCommandName` ```ts knowsCommandName(name: string): boolean ``` | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | string | — | | **Returns** `boolean` #### `knowsEnvName` ```ts knowsEnvName(name: string): boolean ``` | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | string | — | | **Returns** `boolean` #### `knowsCharacterName` ```ts knowsCharacterName(name: string): boolean ``` | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | string | — | | **Returns** `boolean` #### `lookupCharacter` ```ts lookupCharacter(name: string, mode: RuntimeContentMode): CharacterInfo | null ``` | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | string | — | | | `mode` | RuntimeContentMode | — | | **Returns** `CharacterInfo | null` #### `lookupCommand` ```ts lookupCommand(name: string, mode: RuntimeContentMode): CommandInfo | null ``` | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | string | — | | | `mode` | RuntimeContentMode | — | | **Returns** `CommandInfo | null` #### `lookupEnv` ```ts lookupEnv(name: string, mode: RuntimeContentMode): EnvInfo | null ``` | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | string | — | | | `mode` | RuntimeContentMode | — | | **Returns** `EnvInfo | null` #### `lookupExplicitCommand` ```ts lookupExplicitCommand(name: string, mode: RuntimeContentMode): CommandInfo | null ``` | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | string | — | | | `mode` | RuntimeContentMode | — | | **Returns** `CommandInfo | null` ## `KnowledgeBaseOptions` Kind: interface Construction options for an immutable [KnowledgeBase](/api/js/knowledge-base#KnowledgeBase). Omit `packages` to load the default runtime packages, not every package in the catalog. Use `listPackages()` to see the available names; an unknown name throws [TexformConfigError](/api/js/utilities#TexformConfigError). | Name | Type | Description | | --- | --- | --- | | `packages` | string[] | undefined | Package names to load; omit to load the default runtime packages. | | `items` | ContextItem[] | undefined | Custom command/environment/delimiter-control knowledge to inject. | | `removeCommands` | string[] | undefined | Command names to drop from the loaded knowledge. | | `removeEnvironments` | string[] | undefined | Environment names to drop from the loaded knowledge. | | `removeDelimiterControls` | string[] | undefined | Delimiter-control names to drop from the loaded knowledge. | - [KnowledgeBase](https://texform.dev/api/js/knowledge-base.md#KnowledgeBase) - [TexformConfigError](https://texform.dev/api/js/utilities.md#TexformConfigError) ## `DelimiterInfo` Kind: interface | Name | Type | Description | | --- | --- | --- | | `name` | string | | | `isControlSequence` | boolean | | | `allowedMode` | string | | | `unicodeValue` | string | | | `attributes` | { mathvariant: string | null } | | | `package` | string | | ## `ConstructionOptions` Kind: interface Options for the `create*` methods. `mode` is the context mode of the detached subtree. It defaults to text for `createInlineMath`, math for `createDelimitedGroup`, `createScripted`, `createPrime`, and `createInfix`, the group's own mode for `createGroup`, and the document root mode otherwise. | Name | Type | Description | | --- | --- | --- | | `mode` | RuntimeContentMode | undefined | | --- # Document Construct, query, edit, and serialize the editable TeXForm document tree. https://texform.dev/api/js/document ## `Document` Kind: class ```ts Document(options: { knowledgeBase?: KnowledgeBase; mode?: RuntimeContentMode; } | undefined) ``` The editable LaTeX document tree — the working format you read, mutate, serialize, and transform. Reads go through read-only [Node](/api/js/node#Node) handles; edits are validated eagerly and throw [TexformEditError](/api/js/utilities#TexformEditError) on misuse, so no internal failure ever corrupts the tree. A tree that [Document.hasErrors](/api/js/document#Document.hasErrors) is read-only and fixed that way at construction: every editing method then throws. `Document` is a live WASM-backed class instance, not plain data. Documents share an immutable KnowledgeBase with their parser or engine. Transform accepts any complete document with the same knowledge-base identity. ```ts import { Document } from 'texform'; const doc = new Document(); doc.toLatex(); // '' ``` - [Node](https://texform.dev/api/js/node.md#Node) - [TexformEditError](https://texform.dev/api/js/utilities.md#TexformEditError) - [Document.hasErrors](https://texform.dev/api/js/document.md#Document.hasErrors) - [TransformEngine](https://texform.dev/api/js/transform-engine.md#TransformEngine) - [Parsing](https://texform.dev/docs/parsing.md) ### Construction and copying #### `Document` ```ts Document(options: { knowledgeBase?: KnowledgeBase; mode?: RuntimeContentMode; } | undefined) ``` Construct an empty math-mode document holding a single empty root. This is a complete tree (`hasErrors()` is `false`) — the same legal state an empty parse produces. The document is editable and serializable, and it shares the default KnowledgeBase unless one is supplied. | Name | Type | Default | Description | | --- | --- | --- | --- | | `options` | { knowledgeBase?: KnowledgeBase; mode?: RuntimeContentMode; } | undefined | — | | #### `fromSyntax` ```ts fromSyntax(node: SyntaxNode, options: { knowledgeBase?: KnowledgeBase } | undefined): Document ``` Build a document from a [SyntaxNode](/api/js/parser#SyntaxNode) snapshot. Invalid external syntax is rejected rather than corrupting the tree. `fromSyntax` and [Document.toSyntax](/api/js/document#Document.toSyntax) are symmetric over every node kind, including `Error` and `Prime`. A bare `Prime` is a symbol, not a shorthand superscript. The imported document is bound to the supplied KnowledgeBase or the shared default. Throws [TexformConformanceError](/api/js/utilities#TexformConformanceError) if the snapshot does not conform to the knowledge base. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | SyntaxNode | — | - A `SyntaxNode` object, typically produced by `toSyntax()`. | | `options` | { knowledgeBase?: KnowledgeBase } | undefined | — | | **Returns** `Document` The reconstructed editable document. ```ts import { Document, Parser } from 'texform'; const result = new Parser().parse(String.raw`\frac{x}{y}`); if (!result.document) throw new Error('parse failed'); const syntax = result.document.toSyntax(); const doc = Document.fromSyntax(syntax); ``` - [SyntaxNode](https://texform.dev/api/js/parser.md#SyntaxNode) - [Document.toSyntax](https://texform.dev/api/js/document.md#Document.toSyntax) - [TexformConformanceError](https://texform.dev/api/js/utilities.md#TexformConformanceError) #### `clone` ```ts clone(): Document ``` Deep-copy the tree into a new document with new node identities, sharing this document's knowledge base. **Returns** `Document` ### Queries #### `root` ```ts root(): Node ``` Return the root [Node](/api/js/node#Node) of the tree. The root is unique and parentless; it is the document's top-level container node — a `Root` node, not a `Group` — and its children are the top-level content. **Returns** `Node` The root node handle. ```ts const doc = new Parser().parse(String.raw`\frac{x}{y}`).document; doc.root().kind; // 'root' ``` - [Node](https://texform.dev/api/js/node.md#Node) #### `hasErrors` ```ts hasErrors(): boolean ``` Whether the tree contains any `Error` placeholder node. This is a cheap O(1) signal, independent of the [ParseConfig.abortOnError](/api/js/parser#ParseConfig.abortOnError) parse-strictness knob and separate from structural validity. A tree that has errors is read-only. **Returns** `boolean` `true` if any `Error` node is present. ```ts new Parser().parse(String.raw`\sqrt[`).document.hasErrors(); // true ``` - [ParseConfig.abortOnError](https://texform.dev/api/js/parser.md#ParseConfig.abortOnError) #### `isReadOnly` ```ts isReadOnly(): boolean ``` Whether the document rejects edits. Read-only-ness is fixed at construction and is equivalent to [Document.hasErrors](/api/js/document#Document.hasErrors): a tree containing error nodes cannot be edited, so its error count cannot change. **Returns** `boolean` `true` if every editing method will throw. - [Document.hasErrors](https://texform.dev/api/js/document.md#Document.hasErrors) #### `errors` ```ts errors(): Node[] ``` Return the `Error` placeholder [Node](/api/js/node#Node) handles in the tree. **Returns** `Node[]` An array of the `Error` nodes; empty when `hasErrors()` is false. - [Node](https://texform.dev/api/js/node.md#Node) #### `findCommands` ```ts findCommands(name: string): Node[] ``` Return every command [Node](/api/js/node#Node) in the tree whose name equals `name`. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | string | — | - The command name to match, without the leading backslash. | **Returns** `Node[]` The matching command node handles. ```ts const doc = new Parser().parse(String.raw`\frac{x}{y} + \frac{a}{b}`).document; doc.findCommands('frac').length; // 2 ``` - [Node](https://texform.dev/api/js/node.md#Node) #### `findEnvironments` ```ts findEnvironments(name: string): Node[] ``` Return every environment [Node](/api/js/node#Node) in the tree whose name equals `name`. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | string | — | - The environment name to match. | **Returns** `Node[]` The matching environment node handles. - [Node](https://texform.dev/api/js/node.md#Node) #### `nodeSpans` ```ts nodeSpans(): NodeSpanEntry[] ``` Export the parse-time span side table as a list of `{id, span}` entries. Ids follow the parser's tree-path scheme rooted at `root`: `.child.N` for container children, `.arg.N.content` for content-carrying argument slots, `.left` / `.right` for infix operands, `.body` for environment bodies, and `.base` / `.sub` / `.sup` for script slots. Nodes without a recorded span are omitted. Spans reflect the original parse and are not updated by edits. **Returns** `NodeSpanEntry[]` The span entries in tree order. #### `nodeAt` ```ts nodeAt(path: string): Node ``` Resolve a path to a node handle. Paths are rooted and use the scheme of [Node.path](/api/js/node#Node.path); edits shift them, so use paths read from the current tree. Throws [TexformEditError](/api/js/utilities#TexformEditError) if the path does not exist. | Name | Type | Default | Description | | --- | --- | --- | --- | | `path` | string | — | | **Returns** `Node` - [Node.path](https://texform.dev/api/js/node.md#Node.path) - [TexformEditError](https://texform.dev/api/js/utilities.md#TexformEditError) #### `knowledgeBase` ```ts knowledgeBase(): KnowledgeBase ``` The immutable knowledge base bound to this document. Construction and edits are checked against it; `toLatex` is not. **Returns** `KnowledgeBase` ### Output #### `toLatex` ```ts toLatex(options: SerializeOptions | null | undefined): string ``` Serialize the tree back to LaTeX text. The serializer guarantees text idempotency: re-parsing and re-serializing the output yields the same string. `Error` nodes round-trip their captured source snippet verbatim, and pure prime superscripts serialize compactly as `f'` or `f''`. There is intentionally no method named `serialize` on `Document`. For the output styles, see the Serialization guide and [SerializeOptions](/api/js/serialize#SerializeOptions). | Name | Type | Default | Description | | --- | --- | --- | --- | | `options` | SerializeOptions | null | undefined | — | - A [SerializeOptions](/api/js/serialize#SerializeOptions) object, or omit/`null` for the default (spaced) style. | **Returns** `string` The serialized LaTeX string. ```ts const doc = new Parser().parse(String.raw`x^2`).document; doc.toLatex(); // 'x ^ { 2 }' doc.toLatex({ style: 'compact' }); // 'x^2' ``` - [SerializeOptions](https://texform.dev/api/js/serialize.md#SerializeOptions) - [Serialization](https://texform.dev/docs/serialization.md) #### `toTokenizedLatex` ```ts toTokenizedLatex(options: SerializeOptions | null | undefined): TokenizedLatex ``` Serialize the tree to LaTeX together with typed output tokens. Token spans use UTF-8 byte offsets, not JavaScript UTF-16 indices. Empty error snippets produce no zero-width token; use [Document.hasErrors](/api/js/document#Document.hasErrors) to detect whether the document contains error nodes. | Name | Type | Default | Description | | --- | --- | --- | --- | | `options` | SerializeOptions | null | undefined | — | - A [SerializeOptions](/api/js/serialize#SerializeOptions) object, as for [Document.toLatex](/api/js/document#Document.toLatex). `null` / `undefined` / omitted keeps the default (spaced) style. | **Returns** `TokenizedLatex` Serialized LaTeX plus typed fragments from the same traversal. ```ts const tok = new Parser().parse('x^2').document!.toTokenizedLatex(); tok.latex; // 'x ^ { 2 }' tok.tokens[0].kind; // 'character' ``` - [Document.hasErrors](https://texform.dev/api/js/document.md#Document.hasErrors) - [SerializeOptions](https://texform.dev/api/js/serialize.md#SerializeOptions) - [Document.toLatex](https://texform.dev/api/js/document.md#Document.toLatex) #### `toSyntax` ```ts toSyntax(): SyntaxNode ``` Convert the tree to a [SyntaxNode](/api/js/parser#SyntaxNode) object for serde and transport. This is the structured-data channel, distinct from the LaTeX text channel [Document.toLatex](/api/js/document#Document.toLatex). **Returns** `SyntaxNode` The lossless snapshot of the tree. ```ts doc.toSyntax(); // { Root: { ... } } ``` - [SyntaxNode](https://texform.dev/api/js/parser.md#SyntaxNode) - [Document.toLatex](https://texform.dev/api/js/document.md#Document.toLatex) #### `toColumnar` ```ts toColumnar(): ColumnarTree ``` Flatten the tree into columns for bulk structural analysis with Arrow or DataFrame tools. Modes describe node contexts. Missing indices are -1; other missing values are null. Detached subtrees are excluded. Error documents are supported. **Returns** `ColumnarTree` ### Staging constructors The `create*` methods, `parseFragment`, `cloneNode`, and `importNode` build detached subtrees owned by this document. A staged node is not in the tree until you attach it with an edit method. #### `createChar` ```ts createChar(value: string, options: ConstructionOptions | undefined): Node ``` Create a detached single-character node owned by this document. The node is not in the tree until attached with an edit method such as [Document.appendChild](/api/js/document#Document.appendChild). Every `create*` method takes `options.mode` to construct in another context; see [ConstructionOptions](/api/js/knowledge-base#ConstructionOptions) for the defaults. Throws [TexformConformanceError](/api/js/utilities#TexformConformanceError) if the character cannot be written in that mode, such as `\`, `^`, `~`, or a math-mode space, and [TexformEditError](/api/js/utilities#TexformEditError) if the document is read-only. | Name | Type | Default | Description | | --- | --- | --- | --- | | `value` | string | — | - The single character. `"&"` is a literal ampersand written `\&`. | | `options` | ConstructionOptions | undefined | — | | **Returns** `Node` The detached node handle. - [Document.appendChild](https://texform.dev/api/js/document.md#Document.appendChild) - [ConstructionOptions](https://texform.dev/api/js/knowledge-base.md#ConstructionOptions) - [TexformConformanceError](https://texform.dev/api/js/utilities.md#TexformConformanceError) - [TexformEditError](https://texform.dev/api/js/utilities.md#TexformEditError) #### `createText` ```ts createText(value: string, options: ConstructionOptions | undefined): Node ``` Create a detached text node. Throws [TexformConformanceError](/api/js/utilities#TexformConformanceError) if the text cannot be written in the mode. | Name | Type | Default | Description | | --- | --- | --- | --- | | `value` | string | — | | | `options` | ConstructionOptions | undefined | — | | **Returns** `Node` - [TexformConformanceError](https://texform.dev/api/js/utilities.md#TexformConformanceError) #### `createActiveSpace` ```ts createActiveSpace(options: ConstructionOptions | undefined): Node ``` Create a detached active-space node. | Name | Type | Default | Description | | --- | --- | --- | --- | | `options` | ConstructionOptions | undefined | — | | **Returns** `Node` #### `createGroup` ```ts createGroup(mode: RuntimeContentMode, children: (Node | string)[] | undefined, options: ConstructionOptions | undefined): Node ``` Create a detached brace group built in its own `mode`. Throws [TexformParseError](/api/js/utilities#TexformParseError) for source that does not parse cleanly, [TexformConformanceError](/api/js/utilities#TexformConformanceError) for a child built in another mode or an infix with siblings, and [TexformEditError](/api/js/utilities#TexformEditError) for an attached, foreign, or repeated child. | Name | Type | Default | Description | | --- | --- | --- | --- | | `mode` | RuntimeContentMode | — | - The group's content mode and context. | | `children` | (Node | string)[] | undefined | — | - Detached nodes or source strings. A string is parsed in `mode` and its nodes are spliced into the list, so one string may yield several children. | | `options` | ConstructionOptions | undefined | — | | **Returns** `Node` ```ts const group = doc.createGroup('math', ['x^2', doc.createChar('y')]); ``` - [TexformParseError](https://texform.dev/api/js/utilities.md#TexformParseError) - [TexformConformanceError](https://texform.dev/api/js/utilities.md#TexformConformanceError) - [TexformEditError](https://texform.dev/api/js/utilities.md#TexformEditError) #### `createCommand` ```ts createCommand(name: string, args: Arg[] | null | undefined, options: ConstructionOptions | undefined): Node ``` Create a detached prefix command. Throws [TexformConformanceError](/api/js/utilities#TexformConformanceError) if the name is not a prefix command (the message names the right constructor, such as `createDeclarative` for `bf`), the argument count is wrong, or an argument does not fit its slot; [TexformParseError](/api/js/utilities#TexformParseError) for invalid source arguments. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | string | — | - The command name without the leading backslash. A name missing from the knowledge base creates an unknown command without arguments. | | `args` | Arg[] | null | undefined | — | - Values for every signature slot, or only the required slots to omit optional and star slots. See [Arg](/api/js/node#Arg). | | `options` | ConstructionOptions | undefined | — | | **Returns** `Node` ```ts doc.appendChild(doc.root(), doc.createCommand('sqrt', ['x'])); doc.toLatex(); // '\\sqrt { x }' ``` - [Arg](https://texform.dev/api/js/node.md#Arg) - [TexformConformanceError](https://texform.dev/api/js/utilities.md#TexformConformanceError) - [TexformParseError](https://texform.dev/api/js/utilities.md#TexformParseError) #### `createDeclarative` ```ts createDeclarative(name: string, args: Arg[] | null | undefined, options: ConstructionOptions | undefined): Node ``` Create a detached declarative command such as `\bf`. Same conditions as [Document.createCommand](/api/js/document#Document.createCommand), except the record must be declarative. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | string | — | | | `args` | Arg[] | null | undefined | — | | | `options` | ConstructionOptions | undefined | — | | **Returns** `Node` - [Document.createCommand](https://texform.dev/api/js/document.md#Document.createCommand) #### `createEnvironment` ```ts createEnvironment(name: string, args: Arg[] | null | undefined, body: Node | string | (Node | string)[] | null | undefined, options: ConstructionOptions | undefined): Node ``` Create a detached environment. Throws [TexformConformanceError](/api/js/utilities#TexformConformanceError) if an argument does not fit the signature or the body is not a group in the body mode, and [TexformParseError](/api/js/utilities#TexformParseError) for invalid source. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | string | — | - The environment name. An unknown environment takes no arguments and uses the surrounding mode. | | `args` | Arg[] | null | undefined | — | - Slot values, as for [Document.createCommand](/api/js/document#Document.createCommand). | | `body` | Node | string | (Node | string)[] | null | undefined | — | - A detached group in the body mode, a source string parsed in the recorded body mode, a list whose source strings are spliced as in [Document.createGroup](/api/js/document#Document.createGroup), or `null` for an empty body. | | `options` | ConstructionOptions | undefined | — | | **Returns** `Node` - [Document.createCommand](https://texform.dev/api/js/document.md#Document.createCommand) - [Document.createGroup](https://texform.dev/api/js/document.md#Document.createGroup) - [TexformConformanceError](https://texform.dev/api/js/utilities.md#TexformConformanceError) - [TexformParseError](https://texform.dev/api/js/utilities.md#TexformParseError) #### `createAlignmentTab` ```ts createAlignmentTab(options: ConstructionOptions | undefined): Node ``` Create a detached unescaped `&` cell separator. Use `createChar("&")` for a literal ampersand. | Name | Type | Default | Description | | --- | --- | --- | --- | | `options` | ConstructionOptions | undefined | — | | **Returns** `Node` #### `createDelimitedGroup` ```ts createDelimitedGroup(left: string, right: string, children: (Node | string)[] | undefined, options: ConstructionOptions | undefined): Node ``` Create a detached `\left...\right` group, built in math mode by default. Throws [TexformConformanceError](/api/js/utilities#TexformConformanceError) for a delimiter that is neither `.` nor registered in the knowledge base, plus the conditions of `createGroup`. | Name | Type | Default | Description | | --- | --- | --- | --- | | `left` | string | — | - The left delimiter source, such as `"("` or `"\\langle"`. | | `right` | string | — | - The right delimiter source. | | `children` | (Node | string)[] | undefined | — | - Nodes or source strings, spliced as in [Document.createGroup](/api/js/document#Document.createGroup). | | `options` | ConstructionOptions | undefined | — | | **Returns** `Node` - [Document.createGroup](https://texform.dev/api/js/document.md#Document.createGroup) - [TexformConformanceError](https://texform.dev/api/js/utilities.md#TexformConformanceError) #### `createInlineMath` ```ts createInlineMath(children: (Node | string)[] | undefined, options: ConstructionOptions | undefined): Node ``` Create a detached `$...$` group of math children, in a text context by default. Same failure conditions as `createGroup`. | Name | Type | Default | Description | | --- | --- | --- | --- | | `children` | (Node | string)[] | undefined | — | - Nodes or source strings, spliced as in [Document.createGroup](/api/js/document#Document.createGroup). | | `options` | ConstructionOptions | undefined | — | | **Returns** `Node` - [Document.createGroup](https://texform.dev/api/js/document.md#Document.createGroup) #### `createScripted` ```ts createScripted(base: Node | string, sub: Node | string | null | undefined, sup: Node | string | null | undefined, options: ConstructionOptions | undefined): Node ``` Create a detached scripted node, built in math mode by default. Throws [TexformConformanceError](/api/js/utilities#TexformConformanceError) if both scripts are absent or the base is a scripted node, an infix, or an alignment tab, and [TexformParseError](/api/js/utilities#TexformParseError) for invalid source. | Name | Type | Default | Description | | --- | --- | --- | --- | | `base` | Node | string | — | - The base node or source. | | `sub` | Node | string | null | undefined | — | - The subscript node or source, or `null` for none. | | `sup` | Node | string | null | undefined | — | - The superscript node or source, or `null` for none. | | `options` | ConstructionOptions | undefined | — | | **Returns** `Node` ```ts doc.appendChild(doc.root(), doc.createScripted('x', null, '2')); ``` - [TexformConformanceError](https://texform.dev/api/js/utilities.md#TexformConformanceError) - [TexformParseError](https://texform.dev/api/js/utilities.md#TexformParseError) #### `createPrime` ```ts createPrime(count: number, options: ConstructionOptions | undefined): Node ``` Create a detached run of `count` primes, built in math mode by default. Throws [TexformConformanceError](/api/js/utilities#TexformConformanceError) if `count` is not positive. | Name | Type | Default | Description | | --- | --- | --- | --- | | `count` | number | — | | | `options` | ConstructionOptions | undefined | — | | **Returns** `Node` - [TexformConformanceError](https://texform.dev/api/js/utilities.md#TexformConformanceError) #### `createInfix` ```ts createInfix(name: string, left: Node | string, right: Node | string, args: Arg[] | null | undefined, options: ConstructionOptions | undefined): Node ``` Create a detached infix command such as `\over`, built in math mode by default. Throws [TexformConformanceError](/api/js/utilities#TexformConformanceError) under the conditions of `createCommand`, except the record must be infix. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | string | — | | | `left` | Node | string | — | - The left operand node or source. | | `right` | Node | string | — | - The right operand node or source. | | `args` | Arg[] | null | undefined | — | - Values for extra slots, such as the dimension of `\above`. | | `options` | ConstructionOptions | undefined | — | | **Returns** `Node` - [TexformConformanceError](https://texform.dev/api/js/utilities.md#TexformConformanceError) #### `parseFragment` ```ts parseFragment(source: string, options: ConstructionOptions | undefined): Node ``` Parse source into a detached implicit group, in `options.mode` or the root mode. Throws [TexformParseError](/api/js/utilities#TexformParseError) with the diagnostics if the source does not parse cleanly. | Name | Type | Default | Description | | --- | --- | --- | --- | | `source` | string | — | | | `options` | ConstructionOptions | undefined | — | | **Returns** `Node` - [TexformParseError](https://texform.dev/api/js/utilities.md#TexformParseError) #### `cloneNode` ```ts cloneNode(node: Node): Node ``` Deep-copy a subtree of this document as a detached node. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | | **Returns** `Node` #### `importNode` ```ts importNode(node: Node): Node ``` Copy a subtree from another document into this one as a detached node, validating it against this document's knowledge base. Throws [TexformConformanceError](/api/js/utilities#TexformConformanceError) if the subtree does not conform to this knowledge base. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | | **Returns** `Node` - [TexformConformanceError](https://texform.dev/api/js/utilities.md#TexformConformanceError) ### Edit methods All edits are validated eagerly and throw `TexformEditError` on misuse. Nodes must belong to this document, the root cannot be detached, and a tree that `hasErrors()` is read-only. #### `appendChild` ```ts appendChild(parent: Node, child: Node): void ``` Append `child` as the last child of `parent`. Throws [TexformEditError](/api/js/utilities#TexformEditError) if the document is read-only, either node is foreign or missing, `parent` is not a container, or the move would create a cycle. | Name | Type | Default | Description | | --- | --- | --- | --- | | `parent` | Node | — | - The container node to append into. | | `child` | Node | — | - The node to append. | **Returns** `void` - [TexformEditError](https://texform.dev/api/js/utilities.md#TexformEditError) #### `insertChild` ```ts insertChild(parent: Node, index: number, child: Node): void ``` Insert `child` at position `index` among `parent`'s children. Throws [TexformEditError](/api/js/utilities#TexformEditError) if the document is read-only, either node is foreign or missing, `parent` is not a container, `index` is out of range, or the move would create a cycle. | Name | Type | Default | Description | | --- | --- | --- | --- | | `parent` | Node | — | - The container node. | | `index` | number | — | - The zero-based insertion index. | | `child` | Node | — | - The node to insert. | **Returns** `void` - [TexformEditError](https://texform.dev/api/js/utilities.md#TexformEditError) #### `insertBefore` ```ts insertBefore(anchor: Node, node: Node): void ``` Insert `node` immediately before the sibling `anchor`. Throws [TexformEditError](/api/js/utilities#TexformEditError) if the document is read-only, either node is foreign or missing, `anchor` has no parent (such as the root), or the move would create a cycle. | Name | Type | Default | Description | | --- | --- | --- | --- | | `anchor` | Node | — | - The sibling to insert before. | | `node` | Node | — | - The node to insert. | **Returns** `void` - [TexformEditError](https://texform.dev/api/js/utilities.md#TexformEditError) #### `insertAfter` ```ts insertAfter(anchor: Node, node: Node): void ``` Insert `node` immediately after the sibling `anchor`. Same failure conditions as [Document.insertBefore](/api/js/document#Document.insertBefore). | Name | Type | Default | Description | | --- | --- | --- | --- | | `anchor` | Node | — | - The sibling to insert after. | | `node` | Node | — | - The node to insert. | **Returns** `void` - [Document.insertBefore](https://texform.dev/api/js/document.md#Document.insertBefore) #### `replaceWith` ```ts replaceWith(target: Node, replacement: Node): void ``` Replace `target` in place with `replacement`. Throws [TexformEditError](/api/js/utilities#TexformEditError) if the document is read-only, either node is foreign or missing, `target` is the root, or the move would create a cycle. | Name | Type | Default | Description | | --- | --- | --- | --- | | `target` | Node | — | - The node to replace. | | `replacement` | Node | — | - The node to put in its place. | **Returns** `void` - [TexformEditError](https://texform.dev/api/js/utilities.md#TexformEditError) #### `wrap` ```ts wrap(target: Node, wrapper: Node): Node ``` Wrap `target` inside the container `wrapper`. `wrapper` takes `target`'s place in the tree and `target` becomes its child. Throws [TexformEditError](/api/js/utilities#TexformEditError) if the document is read-only, either node is foreign or missing, `wrapper` is not a container, or `target` is the root. | Name | Type | Default | Description | | --- | --- | --- | --- | | `target` | Node | — | - The node to wrap. | | `wrapper` | Node | — | - The container node to wrap it in. | **Returns** `Node` The `wrapper` node, now positioned in the tree. - [TexformEditError](https://texform.dev/api/js/utilities.md#TexformEditError) #### `unwrap` ```ts unwrap(group: Node): Node[] ``` Unwrap a group, splicing its children into its parent in place. Throws [TexformEditError](/api/js/utilities#TexformEditError) if the document is read-only, or `group` is foreign, missing, not a group, or the root. | Name | Type | Default | Description | | --- | --- | --- | --- | | `group` | Node | — | - The group node to dissolve. | **Returns** `Node[]` The freed child node handles, now in the parent. ```ts const doc = new Parser().parse(String.raw`{x y}`).document; const group = doc.root().descendants().find((n) => n.kind === 'group'); doc.unwrap(group); doc.toLatex(); // 'x y' ``` - [TexformEditError](https://texform.dev/api/js/utilities.md#TexformEditError) #### `extract` ```ts extract(node: Node): Node ``` Detach the subtree rooted at `node` and return it as a detached node. The subtree is removed from its parent but kept alive, so it can be re-attached elsewhere. Throws [TexformEditError](/api/js/utilities#TexformEditError) if the document is read-only, or `node` is foreign, missing, or the root. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | - The subtree root to detach. | **Returns** `Node` The detached node. - [TexformEditError](https://texform.dev/api/js/utilities.md#TexformEditError) #### `remove` ```ts remove(node: Node): void ``` Remove the subtree rooted at `node` and discard it. Throws [TexformEditError](/api/js/utilities#TexformEditError) if the document is read-only, or `node` is foreign, missing, or the root. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | - The subtree root to remove. | **Returns** `void` - [TexformEditError](https://texform.dev/api/js/utilities.md#TexformEditError) #### `clear` ```ts clear(node: Node): void ``` Remove every child of the container `node`. Throws [TexformEditError](/api/js/utilities#TexformEditError) if the document is read-only, or `node` is foreign, missing, or not a container. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | - The container to empty. | **Returns** `void` - [TexformEditError](https://texform.dev/api/js/utilities.md#TexformEditError) #### `setText` ```ts setText(node: Node, value: string): void ``` Set the content of a text node. Throws [TexformEditError](/api/js/utilities#TexformEditError) if the document is read-only, or `node` is foreign, missing, or not a text node. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | - The text node to update. | | `value` | string | — | - The new text content. | **Returns** `void` - [TexformEditError](https://texform.dev/api/js/utilities.md#TexformEditError) #### `setChar` ```ts setChar(node: Node, value: string): void ``` Set the character of a char node. Throws [TexformEditError](/api/js/utilities#TexformEditError) if the document is read-only, or `node` is foreign, missing, or not a char node. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | - The char node to update. | | `value` | string | — | - The new single character. | **Returns** `void` - [TexformEditError](https://texform.dev/api/js/utilities.md#TexformEditError) #### `setCommandName` ```ts setCommandName(node: Node, name: string): void ``` Rename a command node. Throws [TexformEditError](/api/js/utilities#TexformEditError) if the document is read-only, or `node` is foreign, missing, or not a command node. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | - The command node to rename. | | `name` | string | — | - The new command name. | **Returns** `void` - [TexformEditError](https://texform.dev/api/js/utilities.md#TexformEditError) #### `setEnvName` ```ts setEnvName(node: Node, name: string): void ``` Rename an environment, keeping its arguments and body. Throws [TexformConformanceError](/api/js/utilities#TexformConformanceError) if the new name's signature or body mode does not fit the existing arguments and body. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | | | `name` | string | — | | **Returns** `void` - [TexformConformanceError](https://texform.dev/api/js/utilities.md#TexformConformanceError) #### `setArg` ```ts setArg(node: Node, index: number, value: Arg): void ``` Set the argument at `index` of a command or environment node. Throws [TexformEditError](/api/js/utilities#TexformEditError) if the document is read-only, `node` is foreign or missing, there is no argument slot at `index`, or `value` does not match the slot shape; [TexformConformanceError](/api/js/utilities#TexformConformanceError) if the value does not fit the slot; [TexformParseError](/api/js/utilities#TexformParseError) for invalid source. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | - The command or environment node. | | `index` | number | — | - The zero-based argument slot index. | | `value` | Arg | — | - The new argument value; see [Arg](/api/js/node#Arg). `null` clears an optional slot. | **Returns** `void` - [Arg](https://texform.dev/api/js/node.md#Arg) - [TexformEditError](https://texform.dev/api/js/utilities.md#TexformEditError) - [TexformConformanceError](https://texform.dev/api/js/utilities.md#TexformConformanceError) - [TexformParseError](https://texform.dev/api/js/utilities.md#TexformParseError) #### `setArgDelimiters` ```ts setArgDelimiters(node: Node, index: number, open: string, close: string): void ``` Change only the boundaries of a filled paired argument. Throws [TexformConformanceError](/api/js/utilities#TexformConformanceError) if the slot is not a filled paired slot or the delimiters are not allowed. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | | | `index` | number | — | | | `open` | string | — | | | `close` | string | — | | **Returns** `void` - [TexformConformanceError](https://texform.dev/api/js/utilities.md#TexformConformanceError) #### `setDelimiters` ```ts setDelimiters(node: Node, left: string, right: string): void ``` Change the delimiters of a `\left...\right` group. Throws [TexformConformanceError](/api/js/utilities#TexformConformanceError) for a delimiter that is neither `.` nor registered. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | | | `left` | string | — | | | `right` | string | — | | **Returns** `void` - [TexformConformanceError](https://texform.dev/api/js/utilities.md#TexformConformanceError) #### `setSubscript` ```ts setSubscript(node: Node, value: Arg | undefined): Node ``` Set or clear the subscript of `node`. A non-scripted node becomes the base of a new scripted node; an existing scripted base is resolved automatically. Throws [TexformConformanceError](/api/js/utilities#TexformConformanceError) if the base cannot carry scripts and [TexformParseError](/api/js/utilities#TexformParseError) for invalid source. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | | | `value` | Arg | undefined | — | - The subscript node or source, or omit/`null` to clear it. | **Returns** `Node` The resulting scripted node, or the base when clearing the last script collapses the wrapper. - [TexformConformanceError](https://texform.dev/api/js/utilities.md#TexformConformanceError) - [TexformParseError](https://texform.dev/api/js/utilities.md#TexformParseError) #### `setSuperscript` ```ts setSuperscript(node: Node, value: Arg | undefined): Node ``` Set or clear the superscript of `node`; behaves like [Document.setSubscript](/api/js/document#Document.setSubscript). | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | | | `value` | Arg | undefined | — | | **Returns** `Node` - [Document.setSubscript](https://texform.dev/api/js/document.md#Document.setSubscript) #### `setPrimeCount` ```ts setPrimeCount(node: Node, count: number): void ``` Set the number of primes in a prime node. Throws [TexformConformanceError](/api/js/utilities#TexformConformanceError) if `count` is not positive. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | Node | — | | | `count` | number | — | | **Returns** `void` - [TexformConformanceError](https://texform.dev/api/js/utilities.md#TexformConformanceError) ### Lifecycle #### `free` ```ts free(): void ``` Release the WASM handle for this document immediately. Prefer a `using` declaration so `[Symbol.dispose]()` runs at scope exit. After `free()`, further use is invalid. Skipping `free()` is safe: garbage collection still reclaims the handle, just not at a determined time. Call `free()` (or use `using`) when many short-lived documents would otherwise accumulate WASM memory. **Returns** `void` #### `[Symbol.dispose]` ```ts [Symbol.dispose](): void ``` Release the WASM handle. Same as [Document.free](/api/js/document#Document.free); used by `using`. **Returns** `void` - [Document.free](https://texform.dev/api/js/document.md#Document.free) ## `NodeSpanEntry` Kind: interface One entry of [Document.nodeSpans](/api/js/document#Document.nodeSpans): a node identifier paired with its recorded source [Span](/api/js/parser#Span). | Name | Type | Description | | --- | --- | --- | | `id` | string | Tree path such as `root.child.0.arg.1.content`. | | `span` | Span | The node's source byte span recorded by the parser. | - [Document.nodeSpans](https://texform.dev/api/js/document.md#Document.nodeSpans) - [Span](https://texform.dev/api/js/parser.md#Span) ## `TokenizedLatex` Kind: interface Serialized LaTeX and tokens produced by the same serializer traversal. | Name | Type | Description | | --- | --- | --- | | `latex` | string | Serialized LaTeX string, identical to [Document.toLatex](/api/js/document#Document.toLatex). | | `tokens` | SerializationToken[] | Ordered, non-overlapping, non-empty fragments. Gaps contain serializer-inserted spacing. Empty error snippets emit no token. | ## `SerializationToken` Kind: interface One non-empty serialization fragment. | Name | Type | Description | | --- | --- | --- | | `text` | string | Fragment text, decoded from bytes `[startByte, endByte)` of the UTF-8 encoding of `latex`. | | `startByte` | number | Inclusive UTF-8 byte offset into [TokenizedLatex.latex](/api/js/document#TokenizedLatex.latex), not a JavaScript UTF-16 index. | | `endByte` | number | Exclusive UTF-8 byte offset into [TokenizedLatex.latex](/api/js/document#TokenizedLatex.latex), not a JavaScript UTF-16 index. | | `kind` | SerializationTokenKind | Semantic category of the fragment. | | `mode` | "math" | "text" | Semantic math/text mode of the fragment, independent of spacing policy. | ## `SerializationTokenKind` Kind: typeAlias ```ts "control_sequence" | "character" | "delimiter" | "text" | "raw" | "error" ``` Closed semantic categories emitted by tokenized serialization. ## `NodeTable` Kind: interface Preorder node columns; field names stay snake_case in both bindings. | Name | Type | Description | | --- | --- | --- | | `kind` | Array<"Root" | "Group" | "Command" | "Infix" | "Declarative" | "Environment" | "Scripted" | "Prime" | "Text" | "Char" | "ActiveSpace" | "AlignmentTab" | "Error"> | | | `parent` | number[] | | | `slot` | Array | | | `slot_index` | number[] | | | `depth` | number[] | | | `mode` | RuntimeContentMode[] | | | `name` | Array | | | `value` | Array | | | `known` | Array | | | `group_kind` | Array<"explicit" | "implicit" | "delimited" | "inline_math" | null> | | | `left` | Array | | | `right` | Array | | ## `ArgumentTable` Kind: interface All argument slots, ordered by owner row and slot index. | Name | Type | Description | | --- | --- | --- | | `owner` | number[] | | | `index` | number[] | | | `form` | Array | | | `present` | boolean[] | | | `value_kind` | Array<"math" | "text" | "operator_name" | "delimiter" | "cs_name" | "dimension" | "integer" | "key_val" | "column" | "boolean" | null> | | | `value` | Array | | | `content` | number[] | | | `open` | Array | | | `close` | Array | | ## `ColumnarTree` Kind: interface Tree columns for bulk structural analysis with Arrow or DataFrame tools. | Name | Type | Description | | --- | --- | --- | | `nodes` | NodeTable | | | `args` | ArgumentTable | | --- # Node Navigate the document tree and read node content through read-only Node handles. https://texform.dev/api/js/node ## `Node` Kind: class A read-only handle into a [Document](/api/js/document#Document). `Node` carries navigation and accessor members but no editing methods — all edits go through the owning document. It is a cheap reference, not a copy of the subtree; reads delegate back into the document. Obtain handles from [Document.root](/api/js/document#Document.root), [Document.errors](/api/js/document#Document.errors), [Document.findCommands](/api/js/document#Document.findCommands), the `create*` staging constructors, and the navigation members below. ```ts const doc = new Parser().parse(String.raw`\frac{x}{y}`).document!; const cmd = doc.root().children()[0]; cmd.kind; // 'command' cmd.isCommand('frac'); // true ``` - [Document](https://texform.dev/api/js/document.md#Document) - [Document.root](https://texform.dev/api/js/document.md#Document.root) - [Document.errors](https://texform.dev/api/js/document.md#Document.errors) - [Document.findCommands](https://texform.dev/api/js/document.md#Document.findCommands) ### Kind #### `kind` The node's kind, read as a property (not a method). Its value is one of the lowercase [NodeKind](/api/js/node#NodeKind) strings (`'root'`, `'group'`, `'command'`, `'scripted'`, ...). ### Predicates #### `isCommand` ```ts isCommand(name: string | null | undefined): boolean ``` Whether the node is a command; with `name`, whether it is that specific command. | Name | Type | Default | Description | | --- | --- | --- | --- | | `name` | string | null | undefined | — | - Optional command name to match; omit/`null` matches any command. | **Returns** `boolean` `true` if the node is a (matching) command. ```ts node.isCommand(); // any command node.isCommand('frac'); // specifically \frac ``` #### `isChar` ```ts isChar(value: string | null | undefined): boolean ``` Whether the node is a char; with `value`, whether it holds that specific character. | Name | Type | Default | Description | | --- | --- | --- | --- | | `value` | string | null | undefined | — | - Optional character to match; omit/`null` matches any char. | **Returns** `boolean` `true` if the node is a (matching) char. #### `isError` ```ts isError(): boolean ``` Whether the node is an `Error` placeholder. **Returns** `boolean` `true` for an `Error` node. #### `isKnown` ```ts isKnown(): boolean | null ``` Whether the knowledge base knows this command or environment; `null` for other kinds. **Returns** `boolean | null` ### Navigation #### `parent` ```ts parent(): Node | null ``` The parent [Node](/api/js/node#Node), or `null` for the root. **Returns** `Node | null` The parent handle, or `null`. - [Node](https://texform.dev/api/js/node.md#Node) #### `children` The child [Node](/api/js/node#Node)s, as a read-only property. #### `nextSibling` ```ts nextSibling(): Node | null ``` The following sibling [Node](/api/js/node#Node), or `null`. **Returns** `Node | null` The next sibling handle, or `null`. - [Node](https://texform.dev/api/js/node.md#Node) #### `prevSibling` ```ts prevSibling(): Node | null ``` The preceding sibling [Node](/api/js/node#Node), or `null`. **Returns** `Node | null` The previous sibling handle, or `null`. - [Node](https://texform.dev/api/js/node.md#Node) #### `ancestors` ```ts ancestors(): Node[] ``` The ancestor [Node](/api/js/node#Node)s from parent upward. **Returns** `Node[]` The ancestor handles, nearest first. - [Node](https://texform.dev/api/js/node.md#Node) #### `descendants` ```ts descendants(): Node[] ``` The descendant [Node](/api/js/node#Node)s in document order. **Returns** `Node[]` The descendant handles. - [Node](https://texform.dev/api/js/node.md#Node) #### `path` ```ts path(): string | null ``` The node's current path from the root, or `null` if detached. Edits change it; [Document.nodeAt](/api/js/document#Document.nodeAt) resolves it. **Returns** `string | null` - [Document.nodeAt](https://texform.dev/api/js/document.md#Document.nodeAt) #### `slot` ```ts slot(): NodeSlot | null ``` The slot this node fills in its parent, or `null` for the root and detached nodes. **Returns** `NodeSlot | null` ### Content accessors #### `commandName` The command name for a `command` / `declarative` node, else `null` (read-only property). #### `envName` The environment name for an `environment` node, else `null` (read-only property). #### `text` The string value for a `text` node, else `null` (read-only property). #### `char` The character for a `char` node, else `null` (read-only property). #### `primeCount` ```ts primeCount(): number | null ``` The prime count (greater than zero) for a `prime` node, else `null`. **Returns** `number | null` The number of consecutive prime symbols, or `null`. `Scripted` determines their binding. ```ts const doc = new Parser().parse(String.raw`f''`).document; const prime = doc.root().descendants().find((n) => n.kind === 'prime'); prime.primeCount(); // 2 ``` #### `errorParts` ```ts errorParts(): { message: string; snippet: string } | null ``` For an `error` node, its captured `{ message, snippet }`, else `null`. **Returns** `{ message: string; snippet: string } | null` The error message and the original source snippet, or `null`. ```ts const doc = new Parser().parse(String.raw`\sqrt[`).document; const err = doc.root().descendants().find((n) => n.isError()); err.errorParts(); // { message: 'unclosed bracket argument', snippet: '\\sqrt[' } ``` #### `contentMode` ```ts contentMode(): RuntimeContentMode | null ``` The runtime content mode (`"math"` or `"text"`) where applicable, else `null`. **Returns** `RuntimeContentMode | null` The node's content mode, or `null`. ### Groups #### `groupKind` ```ts groupKind(): GroupKindRef | null ``` The [GroupKindRef](/api/js/node#GroupKindRef) for a `group` node, else `null`. The result carries a capitalized `kind` discriminator (`'Explicit'`, `'Implicit'`, `'InlineMath'`, `'Delimited'`). A `Delimited` group also carries `left` and `right` delimiter strings such as `'('` or `'\\langle'`. **Returns** `GroupKindRef | null` The group kind view, or `null`. ```ts const doc = new Parser().parse(String.raw`{x}`).document; const group = doc.root().descendants().find((n) => n.kind === 'group'); group.groupKind(); // { kind: 'Explicit' } ``` - [GroupKindRef](https://texform.dev/api/js/node.md#GroupKindRef) ### Command arguments #### `argCount` ```ts argCount(): number ``` The number of argument slots for a command-like node. **Returns** `number` The slot count. #### `arg` ```ts arg(index: number): ArgRef | null ``` The argument at `index`, or `null` if the slot is empty. | Name | Type | Default | Description | | --- | --- | --- | --- | | `index` | number | — | - The zero-based slot index. | **Returns** `ArgRef | null` The [ArgRef](/api/js/node#ArgRef) for the slot, or `null`. ```ts const doc = new Parser().parse(String.raw`\frac{x}{y}`).document; const frac = doc.root().descendants().find((n) => n.isCommand('frac')); frac.argCount(); // 2 frac.arg(0); // { kind: 'Math', node: } ``` - [ArgRef](https://texform.dev/api/js/node.md#ArgRef) #### `argKind` ```ts argKind(index: number): ArgForm | null ``` The delimiter form of the argument slot at `index`, or `null` if there is no such slot. | Name | Type | Default | Description | | --- | --- | --- | --- | | `index` | number | — | | **Returns** `ArgForm | null` #### `argSlots` ```ts argSlots(): Array ``` The full array of argument slots, with `null` for any empty slot. **Returns** `Array` The [ArgRef](/api/js/node#ArgRef) per slot, `null` where empty. - [ArgRef](https://texform.dev/api/js/node.md#ArgRef) ### Scripts and infix #### `scriptBase` ```ts scriptBase(): Node | null ``` The base of a `scripted` node, else `null`. **Returns** `Node | null` The base node handle, or `null`. #### `subscript` ```ts subscript(): Node | null ``` The subscript slot of a `scripted` node, else `null`. **Returns** `Node | null` The subscript node handle, or `null`. ```ts const doc = new Parser().parse(String.raw`x_i^2`).document; const sc = doc.root().descendants().find((n) => n.kind === 'scripted'); [sc.subscript().kind, sc.superscript().kind]; // ['char', 'char'] ``` #### `superscript` ```ts superscript(): Node | null ``` The superscript slot of a `scripted` node, else `null`. **Returns** `Node | null` The superscript node handle, or `null`. #### `infixLeft` ```ts infixLeft(): Node | null ``` The left operand of an `infix` node, else `null`. **Returns** `Node | null` The left operand handle, or `null`. #### `infixRight` ```ts infixRight(): Node | null ``` The right operand of an `infix` node, else `null`. **Returns** `Node | null` The right operand handle, or `null`. #### `envBody` ```ts envBody(): Node | null ``` The body group of an `environment` node, else `null`. **Returns** `Node | null` The body group handle, or `null`. ### Spans #### `span` ```ts span(): Span | null ``` The source byte [Span](/api/js/parser#Span) for the node, or `null` if it has no recorded span. **Returns** `Span | null` The span, or `null`. ```ts const doc = new Parser().parse(String.raw`a + b`).document; doc.root().children[0].span(); // { start: 0, end: 1 } ``` - [Span](https://texform.dev/api/js/parser.md#Span) ### Identity #### `document` ```ts document(): Document ``` **Returns** `Document` #### `isSameNode` ```ts isSameNode(other: Node): boolean ``` | Name | Type | Default | Description | | --- | --- | --- | --- | | `other` | Node | — | | **Returns** `boolean` ### Lifecycle #### `free` ```ts free(): void ``` Release this node handle's WASM wrapper immediately. This does not free the owning [Document](/api/js/document#Document). Prefer a `using` declaration so `[Symbol.dispose]()` runs at scope exit. After `free()`, further use of this handle is invalid. Skipping `free()` is safe: garbage collection still reclaims the wrapper. **Returns** `void` - [Document](https://texform.dev/api/js/document.md#Document) #### `[Symbol.dispose]` ```ts [Symbol.dispose](): void ``` Release this handle. Same as [Node.free](/api/js/node#Node.free); used by `using`. **Returns** `void` - [Node.free](https://texform.dev/api/js/node.md#Node.free) ## `NodeKind` Kind: typeAlias ```ts "root" | "group" | "command" | "infix" | "declarative" | "environment" | "scripted" | "prime" | "text" | "char" | "activeSpace" | "alignmentTab" | "error" ``` The runtime kind of a [Node](/api/js/node#Node), read from the lowercase [Node.kind](/api/js/node#Node.kind) property. Note the casing convention: node kinds are lowercase strings (`'command'`, `'scripted'`), unlike the capitalized discriminators inside argument and group views (`arg.kind === 'Math'`, `groupKind().kind === 'Explicit'`). - [Node](https://texform.dev/api/js/node.md#Node) - [Node.kind](https://texform.dev/api/js/node.md#Node.kind) ## `SyntaxContentMode` Kind: typeAlias ```ts "Math" | "Text" ``` Content mode as it appears in a [SyntaxNode](/api/js/parser#SyntaxNode) snapshot: `"Math"` or `"Text"` (capitalized snapshot form). - [SyntaxNode](https://texform.dev/api/js/parser.md#SyntaxNode) - [RuntimeContentMode](https://texform.dev/api/js/node.md#RuntimeContentMode) ## `RuntimeContentMode` Kind: typeAlias ```ts "math" | "text" ``` Content mode as reported by the runtime tree API: `"math"` or `"text"` (lowercase form used by [Node.contentMode](/api/js/node#Node.contentMode) and the `mode` arguments of lookup methods). - [Node.contentMode](https://texform.dev/api/js/node.md#Node.contentMode) - [SyntaxContentMode](https://texform.dev/api/js/node.md#SyntaxContentMode) ## `GroupKind` Kind: typeAlias ```ts "Explicit" | "Implicit" | { Delimited: { left: Delimiter; right: Delimiter } } | "InlineMath" ``` The kind of a group in [SyntaxNode](/api/js/parser#SyntaxNode) snapshot form: an `Explicit` brace group, an `Implicit` group, a `Delimited` group carrying its `left`/`right` delimiters, or an `InlineMath` group. - [SyntaxNode](https://texform.dev/api/js/parser.md#SyntaxNode) - [GroupKindRef](https://texform.dev/api/js/node.md#GroupKindRef) ## `GroupKindRef` Kind: typeAlias ```ts { kind: "Explicit" } | { kind: "Implicit" } | { kind: "Delimited"; left: string; right: string } | { kind: "InlineMath" } ``` The kind of a group in runtime view form, with a `kind` discriminator. Returned by [Node.groupKind](/api/js/node#Node.groupKind); the discriminator keeps its capitalized form (`'Explicit'`, `'Delimited'`, ...). A `Delimited` group carries `left` and `right` delimiter strings. - [Node.groupKind](https://texform.dev/api/js/node.md#Node.groupKind) - [GroupKind](https://texform.dev/api/js/node.md#GroupKind) ## `Delimiter` Kind: typeAlias ```ts "None" | { Char: string } | { Control: string } ``` A delimiter token in [SyntaxNode](/api/js/parser#SyntaxNode) snapshot form: the literal `"None"`, a single character (`{ Char }`), or a control sequence (`{ Control }`). Runtime views use delimiter strings instead: `.`, one character, or a backslash-prefixed control sequence such as `\langle`. - [SyntaxNode](https://texform.dev/api/js/parser.md#SyntaxNode) ## `Argument` Kind: interface A present command/environment argument in [SyntaxNode](/api/js/parser#SyntaxNode) snapshot form, pairing its syntactic [ArgumentKind](/api/js/node#ArgumentKind) with its parsed [ArgumentValue](/api/js/node#ArgumentValue). | Name | Type | Description | | --- | --- | --- | | `kind` | ArgumentKind | The argument's syntactic kind (mandatory, optional, delimited, ...). | | `no_leading_space` | boolean | undefined | Whether the argspec slot was prefixed with `!`. | | `value` | ArgumentValue | The argument's parsed value. | - [SyntaxNode](https://texform.dev/api/js/parser.md#SyntaxNode) - [ArgumentKind](https://texform.dev/api/js/node.md#ArgumentKind) - [ArgumentValue](https://texform.dev/api/js/node.md#ArgumentValue) ## `ArgumentKind` Kind: typeAlias ```ts "Mandatory" | "Optional" | "Star" | "Group" | { Until: { close: Delimiter } } | { Delimited: { open: Delimiter; close: Delimiter } } | { Paired: { open: Delimiter; close: Delimiter } } ``` The syntactic kind of a command argument in [SyntaxNode](/api/js/parser#SyntaxNode) snapshot form (mandatory, optional, starred, group, until, or delimited/paired). Until forms carry `close`; delimited/paired forms carry `open` and `close`. - [SyntaxNode](https://texform.dev/api/js/parser.md#SyntaxNode) - [ArgSpecFormInfo](https://texform.dev/api/js/parser.md#ArgSpecFormInfo) ## `ArgumentValue` Kind: typeAlias ```ts { MathContent: SyntaxNode } | { TextContent: SyntaxNode } | { OperatorNameContent: SyntaxNode } | { Delimiter: Delimiter } | { CSName: string } | { Dimension: string } | { Integer: string } | { KeyVal: string } | { Column: string } | { Boolean: boolean } ``` The parsed value carried by an [Argument](/api/js/node#Argument) in snapshot form: math or text content, a delimiter, a control-sequence name, a dimension, an integer, a key=value list, a column spec, or a boolean flag. - [Argument](https://texform.dev/api/js/node.md#Argument) - [ArgRef](https://texform.dev/api/js/node.md#ArgRef) ## `ArgumentSlot` Kind: typeAlias ```ts Argument | null | undefined ``` An argument slot of a [Node](/api/js/node#Node)'s argument list: either a present [Argument](/api/js/node#Argument) or an empty slot (`null`/`undefined`). Optional argument slots that were not supplied in the source surface as the empty value rather than being dropped, so a slot's index stays stable. - [Node](https://texform.dev/api/js/node.md#Node) - [Argument](https://texform.dev/api/js/node.md#Argument) ## `ArgRef` Kind: typeAlias ```ts { form: ArgForm } & ( | { kind: "Math"; node: Node } | { kind: "Text"; node: Node } | { kind: "OperatorName"; node: Node } | { kind: "Delimiter"; value: string } | { kind: "CSName"; value: string } | { kind: "Dimension"; value: string } | { kind: "Integer"; value: string } | { kind: "KeyVal"; value: string } | { kind: "Column"; value: string } | { kind: "Boolean"; value: boolean }) ``` A command/environment argument in runtime view form, returned by [Node.arg](/api/js/node#Node.arg) and [Node.argSlots](/api/js/node#Node.argSlots). Content kinds (`Math`, `Text`, `OperatorName`) carry a live `node` [Node](/api/js/node#Node) handle; leaf kinds (`Delimiter`, `CSName`, `Dimension`, `Integer`, `KeyVal`, `Column`, `Boolean`) carry a `value`. `Delimiter` values use source strings. The `kind` discriminator keeps its capitalized form. - [Node.arg](https://texform.dev/api/js/node.md#Node.arg) - [Node.argSlots](https://texform.dev/api/js/node.md#Node.argSlots) - [Node](https://texform.dev/api/js/node.md#Node) - [ArgForm](https://texform.dev/api/js/node.md#ArgForm) ## `ArgForm` Kind: typeAlias ```ts { kind: "mandatory" | "optional" | "star" | "group" | "until" | "delimited" | "paired"; open?: string; close?: string } ``` The delimiter form of an argument slot; `close` is present for `until`, and both `open` and `close` for `delimited` and `paired` forms. ## `NodeSlot` Kind: typeAlias ```ts { kind: "child" | "arg" | "script_base" | "subscript" | "superscript" | "infix_left" | "infix_right" | "env_body"; index: number | null } ``` The parent slot a [Node](/api/js/node#Node) fills; `index` is set for indexed slots such as children and arguments. - [Node](https://texform.dev/api/js/node.md#Node) ## `Arg` Kind: typeAlias ```ts Node | string | boolean | null | { value: Arg; open: string; close: string } ``` A slot value accepted by construction and editing methods: a `string` is the source written inside the slot's boundaries, a detached [Node](/api/js/node#Node) is content, `boolean` sets a star slot, `null` omits an optional slot, and `{ value, open, close }` selects an explicit delimiter pair. - [Node](https://texform.dev/api/js/node.md#Node) --- # TransformEngine Normalize formulas with a profile, tune the pipeline, and read the transform report. https://texform.dev/api/js/transform-engine ## `TransformEngine` Kind: class ```ts TransformEngine(options: TransformEngineOptions) ``` A profile-based normalization engine: a parser paired with a transform pipeline. `TransformEngine` normalizes a formula into the canonical form selected by a [Profile](/api/js/transform-engine#Profile). It exposes a string-to-string [TransformEngine.normalize](/api/js/transform-engine#TransformEngine.normalize) path and an in-place [TransformEngine.transform](/api/js/transform-engine#TransformEngine.transform) path over a live [Document](/api/js/document#Document). It also exposes its own [TransformEngine.parse](/api/js/transform-engine#TransformEngine.parse) and the shared knowledge base through [TransformEngine.knowledgeBase](/api/js/transform-engine#TransformEngine.knowledgeBase). For the conceptual model — profiles, the multi-phase pipeline, and the eliminated-form contract — see the Transforms guide. Since 0.4.0 the bundled parser defaults to lenient [ParseConfig](/api/js/parser#ParseConfig), and transform options are a nested camelCase overlay ([TransformConfig](/api/js/transform-engine#TransformConfig) / [NormalizeConfig](/api/js/transform-engine#NormalizeConfig)), not flat `rewriteEnabled` / `maxIterations` keys. `null` / `undefined` / omitted means not set. Unknown keys, snake_case keys, arrays in object positions, and wrong scalar types throw [TexformConfigError](/api/js/utilities#TexformConfigError) with a camelCase path. There is no complete config class on the JavaScript surface; read the effective defaults with [TransformEngine.defaultTransformConfig](/api/js/transform-engine#TransformEngine.defaultTransformConfig). ```ts import { TransformEngine } from 'texform'; const engine = new TransformEngine({ profile: 'corpus' }); engine.normalize(String.raw`a \over b`); // '\\frac { a } { b }' engine.normalize(String.raw`a \over b`, { rewrite: { enabled: false } }); engine.normalizeWithReport(String.raw`a \over b`).report.rewrite.iterations; ``` - [Profile](https://texform.dev/api/js/transform-engine.md#Profile) - [TransformEngine.normalize](https://texform.dev/api/js/transform-engine.md#TransformEngine.normalize) - [TransformEngine.transform](https://texform.dev/api/js/transform-engine.md#TransformEngine.transform) - [Document](https://texform.dev/api/js/document.md#Document) - [TransformEngine.parse](https://texform.dev/api/js/transform-engine.md#TransformEngine.parse) - [TransformEngine.knowledgeBase](https://texform.dev/api/js/transform-engine.md#TransformEngine.knowledgeBase) - [ParseConfig](https://texform.dev/api/js/parser.md#ParseConfig) - [TransformConfig](https://texform.dev/api/js/transform-engine.md#TransformConfig) - [NormalizeConfig](https://texform.dev/api/js/transform-engine.md#NormalizeConfig) - [TexformConfigError](https://texform.dev/api/js/utilities.md#TexformConfigError) - [TransformEngine.defaultTransformConfig](https://texform.dev/api/js/transform-engine.md#TransformEngine.defaultTransformConfig) - [Parser](https://texform.dev/api/js/parser.md#Parser) - [Transforms](https://texform.dev/docs/transforms.md) #### `TransformEngine` ```ts TransformEngine(options: TransformEngineOptions) ``` Construct an engine for a profile, optionally sharing a knowledge base, or disabling rules. | Name | Type | Default | Description | | --- | --- | --- | --- | | `options` | TransformEngineOptions | — | - A [TransformEngineOptions](/api/js/transform-engine#TransformEngineOptions) object. `profile` is required; an unknown profile throws [TexformConfigError](/api/js/utilities#TexformConfigError). | #### `parse` ```ts parse(src: string, config: ParseConfig | null | undefined): ParseResult ``` Parse a LaTeX string into a [ParseResult](/api/js/parser#ParseResult), using this engine's parser. Non-null documents from this result share the engine's KnowledgeBase, so they can be edited and then passed to [TransformEngine.transform](/api/js/transform-engine#TransformEngine.transform). The The engine parser defaults to lenient parsing; pass `config` to override it per call. | Name | Type | Default | Description | | --- | --- | --- | --- | | `src` | string | — | - The LaTeX source string. | | `config` | ParseConfig | null | undefined | — | - A [ParseConfig](/api/js/parser#ParseConfig) overlay, or omit/`null` for the defaults. | **Returns** `ParseResult` The parse result (`document` and `diagnostics`). - [ParseResult](https://texform.dev/api/js/parser.md#ParseResult) - [TransformEngine.transform](https://texform.dev/api/js/transform-engine.md#TransformEngine.transform) - [ParseConfig](https://texform.dev/api/js/parser.md#ParseConfig) #### `normalize` ```ts normalize(src: string, config: NormalizeConfig | null | undefined): string ``` Parse, normalize, and serialize a formula in one call. Normalization is gated on a complete tree: if the input cannot produce one, this throws [TexformParseError](/api/js/utilities#TexformParseError) carrying its `diagnostics` and the partial `document`. Empty input is complete and normalizes normally. | Name | Type | Default | Description | | --- | --- | --- | --- | | `src` | string | — | - The LaTeX source string. | | `config` | NormalizeConfig | null | undefined | — | - A [NormalizeConfig](/api/js/transform-engine#NormalizeConfig) overlay, or omit/`null` to use the engine defaults. The same `{ rewrite: { enabled: false } }` shape is accepted by [TransformEngine.transform](/api/js/transform-engine#TransformEngine.transform). | **Returns** `string` The serialized LaTeX string. This path does not build a report. ```ts const engine = new TransformEngine({ profile: 'corpus' }); engine.normalize(String.raw`\dv{f}{x}`); // '\\frac { \\mathrm { d } f } { \\mathrm { d } x }' ``` - [TexformParseError](https://texform.dev/api/js/utilities.md#TexformParseError) - [NormalizeConfig](https://texform.dev/api/js/transform-engine.md#NormalizeConfig) - [TransformEngine.transform](https://texform.dev/api/js/transform-engine.md#TransformEngine.transform) #### `normalizeWithReport` ```ts normalizeWithReport(src: string, config: NormalizeConfig | null | undefined): NormalizeReportResult ``` Parse, normalize, and serialize a formula, returning text and a diagnostic report. Accepts the same source and [NormalizeConfig](/api/js/transform-engine#NormalizeConfig) overlay as [TransformEngine.normalize](/api/js/transform-engine#TransformEngine.normalize). Output text and errors follow that contract. Report fields, phase divisions, and counters are diagnostic and are not part of the stable compatibility promise. A failure does not return a partial report. | Name | Type | Default | Description | | --- | --- | --- | --- | | `src` | string | — | - The LaTeX source string. | | `config` | NormalizeConfig | null | undefined | — | - A [NormalizeConfig](/api/js/transform-engine#NormalizeConfig) overlay, or omit/`null` to use the engine defaults. | **Returns** `NormalizeReportResult` The normalized string and its [TransformReport](/api/js/transform-engine#TransformReport). - [NormalizeConfig](https://texform.dev/api/js/transform-engine.md#NormalizeConfig) - [TransformEngine.normalize](https://texform.dev/api/js/transform-engine.md#TransformEngine.normalize) - [TransformReport](https://texform.dev/api/js/transform-engine.md#TransformReport) #### `transform` ```ts transform(document: Document, config: TransformConfig | null | undefined): void ``` Transform a live [Document](/api/js/document#Document) in place. The document must share this engine's KnowledgeBase instance. A document that [Document.hasErrors](/api/js/document#Document.hasErrors) is read-only and cannot be transformed; this precondition error is surfaced as [TexformTransformError](/api/js/utilities#TexformTransformError). This path returns no value and does not build a report. | Name | Type | Default | Description | | --- | --- | --- | --- | | `document` | Document | — | - The live document to update in place. | | `config` | TransformConfig | null | undefined | — | - A [TransformConfig](/api/js/transform-engine#TransformConfig) overlay, or omit/`null` to use the profile defaults. Nested per-phase shape; does not accept parse options. | **Returns** `void` ```ts const engine = new TransformEngine({ profile: 'corpus' }); const result = engine.parse(String.raw`a \over b`); if (result.document) { engine.transform(result.document); result.document.toLatex(); // '\\frac { a } { b }' } ``` - [Document](https://texform.dev/api/js/document.md#Document) - [Document.hasErrors](https://texform.dev/api/js/document.md#Document.hasErrors) - [TexformTransformError](https://texform.dev/api/js/utilities.md#TexformTransformError) - [TransformConfig](https://texform.dev/api/js/transform-engine.md#TransformConfig) #### `transformWithReport` ```ts transformWithReport(document: Document, config: TransformConfig | null | undefined): TransformReport ``` Transform a live [Document](/api/js/document#Document) in place and return its diagnostic report. Accepts the same document and [TransformConfig](/api/js/transform-engine#TransformConfig) overlay as [TransformEngine.transform](/api/js/transform-engine#TransformEngine.transform). Ownership, completeness, and errors follow that contract. The report is diagnostic and is not part of the stable compatibility promise. A failure does not return a partial report. | Name | Type | Default | Description | | --- | --- | --- | --- | | `document` | Document | — | - The live document to update in place. | | `config` | TransformConfig | null | undefined | — | - A [TransformConfig](/api/js/transform-engine#TransformConfig) overlay, or omit/`null` to use the profile defaults. | **Returns** `TransformReport` The diagnostic transform report. ```ts const engine = new TransformEngine({ profile: 'corpus' }); const result = engine.parse(String.raw`a \over b`); if (result.document) { const report = engine.transformWithReport(result.document); report.rewrite.rules.some((rule) => rule.key === 'base/over-to-frac'); } ``` - [Document](https://texform.dev/api/js/document.md#Document) - [TransformConfig](https://texform.dev/api/js/transform-engine.md#TransformConfig) - [TransformEngine.transform](https://texform.dev/api/js/transform-engine.md#TransformEngine.transform) #### `defaultParseConfig` ```ts defaultParseConfig(): Complete ``` Return this engine's complete default [ParseConfig](/api/js/parser#ParseConfig). **Returns** `Complete` - [ParseConfig](https://texform.dev/api/js/parser.md#ParseConfig) #### `defaultTransformConfig` ```ts defaultTransformConfig(): Complete ``` Return this engine's complete default [TransformConfig](/api/js/transform-engine#TransformConfig) for its profile. **Returns** `Complete` - [TransformConfig](https://texform.dev/api/js/transform-engine.md#TransformConfig) ### Knowledge-base lookups Query the knowledge the parser loaded. Predicate methods take a single `name`; lookup methods take `name` and a `mode` of `"math"` or `"text"` and return a knowledge-entry object or `null`. #### `knowledgeBase` ```ts knowledgeBase(): KnowledgeBase ``` **Returns** `KnowledgeBase` ### Lifecycle #### `free` ```ts free(): void ``` Release the WASM handle for this engine immediately. Prefer a `using` declaration so `[Symbol.dispose]()` runs at scope exit. After `free()`, further use is invalid. Skipping `free()` is safe: garbage collection still reclaims the handle, just not at a determined time. Call `free()` (or use `using`) when many short-lived engines would otherwise accumulate WASM memory. **Returns** `void` #### `[Symbol.dispose]` ```ts [Symbol.dispose](): void ``` Release the WASM handle. Same as [TransformEngine.free](/api/js/transform-engine#TransformEngine.free); used by `using`. **Returns** `void` - [TransformEngine.free](https://texform.dev/api/js/transform-engine.md#TransformEngine.free) ## `TransformEngineOptions` Kind: interface Construction options for [TransformEngine](/api/js/transform-engine#TransformEngine): a shared knowledge base, the required normalization [Profile](/api/js/transform-engine#Profile), and optional rule disabling. | Name | Type | Description | | --- | --- | --- | | `profile` | Profile | The normalization profile (required). Unknown values throw [TexformConfigError](/api/js/utilities#TexformConfigError). | | `disableRules` | string[] | undefined | Rewrite rule keys to disable, such as `"physics/dv-to-frac-d"`. | - [TransformEngine](https://texform.dev/api/js/transform-engine.md#TransformEngine) - [Profile](https://texform.dev/api/js/transform-engine.md#Profile) ## `NormalizeConfig` Kind: interface Per-run overlay for [TransformEngine.normalize](/api/js/transform-engine#TransformEngine.normalize): parse keys from [ParseConfig](/api/js/parser#ParseConfig) plus transform keys from [TransformConfig](/api/js/transform-engine#TransformConfig), flattened together. `null` / `undefined` / omitted means not set. ```ts const engine = new TransformEngine({ profile: 'corpus' }); engine.normalize(String.raw`a \over b`, { rewrite: { maxIterations: 50 }, flattenGroups: { enabled: false }, }); ``` - [TransformEngine.normalize](https://texform.dev/api/js/transform-engine.md#TransformEngine.normalize) - [ParseConfig](https://texform.dev/api/js/parser.md#ParseConfig) - [TransformConfig](https://texform.dev/api/js/transform-engine.md#TransformConfig) ## `TransformConfig` Kind: interface The nested, per-phase transform configuration, overriding a profile's transform defaults. Each field controls one pipeline phase. This is the transform-only shape; it does not accept parser-strictness keys (those belong to [NormalizeConfig](/api/js/transform-engine#NormalizeConfig)). Every field is an optional overlay key: `null` / `undefined` / omitted keeps the profile's value. Unknown keys, snake_case keys, arrays in object positions, and wrong scalar types throw [TexformConfigError](/api/js/utilities#TexformConfigError) with a camelCase field path. Read the effective defaults with [TransformEngine.defaultTransformConfig](/api/js/transform-engine#TransformEngine.defaultTransformConfig). | Name | Type | Description | | --- | --- | --- | | `lowerAttributes` | LowerAttributesConfig | undefined | LowerAttributes phase switches. | | `rewrite` | RewriteConfig | undefined | Rewrite phase switches. | | `finalizeAst` | FinalizeAstConfig | undefined | FinalizeAst phase switches. | | `flattenGroups` | FlattenGroupsConfig | null | undefined | FlattenGroups phase switches. `null` keeps the profile's FlattenGroups values. | - [NormalizeConfig](https://texform.dev/api/js/transform-engine.md#NormalizeConfig) - [TexformConfigError](https://texform.dev/api/js/utilities.md#TexformConfigError) - [TransformEngine.defaultTransformConfig](https://texform.dev/api/js/transform-engine.md#TransformEngine.defaultTransformConfig) ## `RewriteConfig` Kind: interface Per-run switches for the fixed-point Rewrite phase. | Name | Type | Description | | --- | --- | --- | | `enabled` | boolean | undefined | Whether the phase runs. When `false`, legacy syntax is left untouched. | | `maxIterations` | number | undefined | Cap on Rewrite fixed-point passes. | - [TransformConfig](https://texform.dev/api/js/transform-engine.md#TransformConfig) ## `LowerAttributesConfig` Kind: interface Per-run switches for the LowerAttributes phase. | Name | Type | Description | | --- | --- | --- | | `enabled` | boolean | undefined | Whether the phase runs. Defaults to the profile's setting. | - [TransformConfig](https://texform.dev/api/js/transform-engine.md#TransformConfig) ## `FinalizeAstConfig` Kind: interface Per-run switches for the FinalizeAst phase. | Name | Type | Description | | --- | --- | --- | | `enabled` | boolean | undefined | Whether the phase runs. Defaults to enabled in every public profile. | - [TransformConfig](https://texform.dev/api/js/transform-engine.md#TransformConfig) ## `FlattenGroupsConfig` Kind: interface Per-run switches for the FlattenGroups phase (redundant-brace removal). `enabled` governs whether the phase runs. `preserveRenderedSpacing` keeps groups whose only public-facing effect is rendered math spacing; it does not control serializer source whitespace ([SerializeOptions](/api/js/serialize#SerializeOptions) `*Spacing` fields). Structural guards stay on even when `preserveRenderedSpacing` is `false`. Omitted keys fall back to the profile's defaults — for example, `corpus` leaves `preserveRenderedSpacing` off. | Name | Type | Description | | --- | --- | --- | | `enabled` | boolean | null | undefined | Whether the phase runs. `null` / omitted keeps the profile value. | | `preserveRenderedSpacing` | boolean | null | undefined | Keep groups whose only public-facing effect is rendered math spacing. This is not serializer source spacing. Structural guards stay on even when this is `false`. `null` / omitted keeps the profile value. | - [SerializeOptions](https://texform.dev/api/js/serialize.md#SerializeOptions) - [TransformConfig](https://texform.dev/api/js/transform-engine.md#TransformConfig) ## `NormalizeReportResult` Kind: interface Text plus the diagnostic report from [TransformEngine.normalizeWithReport](/api/js/transform-engine#TransformEngine.normalizeWithReport). Plain [TransformEngine.normalize](/api/js/transform-engine#TransformEngine.normalize) returns the string directly and does not build this object. | Name | Type | Description | | --- | --- | --- | | `normalized` | string | The serialized LaTeX after parsing and normalization. | | `report` | TransformReport | The diagnostic report for this call. | - [TransformEngine.normalizeWithReport](https://texform.dev/api/js/transform-engine.md#TransformEngine.normalizeWithReport) - [TransformEngine.normalize](https://texform.dev/api/js/transform-engine.md#TransformEngine.normalize) ## `TransformReport` Kind: interface Diagnostic report for one explicit report call. Output text and errors follow the ordinary transform contract. Report fields, phase divisions, and these counters are diagnostic: they are not part of the stable compatibility promise. A disabled phase stays present as zeros or an empty array. Field names are camelCase, the JavaScript view of the snake_case data exposed by the Python binding. | Name | Type | Description | | --- | --- | --- | | `warnings` | TransformWarning[] | Unknown input names, deduplicated and sorted by kind and name. | | `lowerAttributes` | LowerAttributesReport | LowerAttributes activity, summed across its two passes. | | `rewrite` | RewriteReport | Rewrite iterations and per-rule outcomes. | | `finalizeAst` | FinalizeAstReport | Prime merges and text normalizations. | | `flattenGroups` | FlattenGroupsReport | Flatten actions and guard hits. | ## `TransformWarning` Kind: interface An unknown input name whose arguments or scope cannot be guaranteed. | Name | Type | Description | | --- | --- | --- | | `kind` | "unknown-command" | "unknown-environment" | The unknown input name's category. | | `name` | string | Name without command or environment delimiters. | | `message` | string | Explanation of the semantic uncertainty. | ## `LowerAttributesReport` Kind: interface The LowerAttributes phase report. | Name | Type | Description | | --- | --- | --- | | `attributes` | AttributeStat[] | Per-attribute activity, sorted by `attr` then `value`. | | `eliminatedEmptySegments` | number | Empty attribute segments removed by the phase. | ## `AttributeStat` Kind: interface Activity for one `(attr, value)` pair in LowerAttributes. | Name | Type | Description | | --- | --- | --- | | `attr` | string | Stable attribute name, such as `"math_font"` or `"text_size"`. | | `value` | string | Stable value token for that attribute. | | `consumed` | AttributeFormCounts | Input forms consumed by LowerAttributes. | | `redundant` | AttributeFormCounts | Consumed forms that did not change the emitted attribute state. | | `emitted` | AttributeFormCounts | Canonical forms emitted by LowerAttributes. | ## `AttributeFormCounts` Kind: interface Counts of an attribute form split by carrier: `declaratives` (scope-affecting declarations such as `{\bf ...}`) and `prefixes` (prefix commands such as `\mathbf{...}`). | Name | Type | Description | | --- | --- | --- | | `declaratives` | number | Count carried by declarative-form markup. | | `prefixes` | number | Count carried by prefix-command markup. | ## `RewriteRuleStat` Kind: interface One rewrite rule's outcome for a single call. | Name | Type | Description | | --- | --- | --- | | `key` | string | Stable rule key. | | `appliedCount` | number | Times the rule applied. | | `skippedCount` | number | Times the scheduler matched the rule and apply returned skipped. | ## `RewriteReport` Kind: interface The Rewrite phase report. | Name | Type | Description | | --- | --- | --- | | `iterations` | number | Fixed-point passes, including the final unchanged convergence check. Zero when the phase is disabled or no rules are scheduled. | | `rules` | RewriteRuleStat[] | Rules that recorded activity, sorted by `key`. | ## `FinalizeAstReport` Kind: interface The FinalizeAst phase report. | Name | Type | Description | | --- | --- | --- | | `primeRunMerges` | number | Contiguous `Prime` runs merged. Each run counts once. | | `textNormalizations` | number | Text segments or text argument slots whose content changed. | ## `FlattenGroupsActionCounts` Kind: interface Flatten actions that changed the tree. | Name | Type | Description | | --- | --- | --- | | `removedEmpty` | number | Empty groups removed. | | `replacedSingleChild` | number | Groups replaced by their only child. | | `inlinedMultiChild` | number | Multi-child groups inlined into the parent. | | `unwrappedSlot` | number | Slot wrappers removed. | ## `FlattenGroupsGuardCounts` Kind: interface Preserve-guard hits for one FlattenGroups run. A hit is the first guard branch reached for a group, plus the scripted-base sub-count. `commandContactViaScriptedBase` is also included in `commandContact`. These counters are not a partition of preserved groups. | Name | Type | Description | | --- | --- | --- | | `unknownCommandArguments` | number | Possible arguments following an unknown command. | | `argumentCapture` | number | Groups whose removal would let a command before or inside them read the next token as an omitted optional or delimited argument. | | `declarativeScope` | number | Declarative-scope protection. | | `scriptBase` | number | Script-base protection. | | `envBody` | number | Environment-body protection. | | `infixScope` | number | Infix-scope protection. | | `commandContact` | number | Command-contact protection. | | `commandContactViaScriptedBase` | number | Command contact established through a scripted base. | | `emptyGroup` | number | Empty-group protection. | | `loneAtomSpacingChar` | number | Single spacing-character protection. | | `leadingAtomSpacingChar` | number | Leading spacing-character protection. | | `delimitedPair` | number | Delimited-pair protection. | ## `FlattenGroupsReport` Kind: interface The FlattenGroups phase report. | Name | Type | Description | | --- | --- | --- | | `actions` | FlattenGroupsActionCounts | Flatten actions that changed the tree. | | `guardHits` | FlattenGroupsGuardCounts | Preserve-guard hits for this call. | ## `TransformProfile` Kind: typeAlias ```ts "authoring" | "faithful" | "corpus" | "equiv" ``` Normalization profile selecting the canonical form for one downstream scenario: `authoring` (polished author-facing output), `faithful` (same rendered formula, convenience macros expanded), `corpus` (training-data normalization, layout hints dropped), or `equiv` (aggressive canonicalization for equivalence comparison). - [TransformEngineOptions](https://texform.dev/api/js/transform-engine.md#TransformEngineOptions) ## `Profile` Kind: typeAlias ```ts "authoring" | "faithful" | "corpus" | "equiv" ``` Normalization profile passed to [TransformEngineOptions](/api/js/transform-engine#TransformEngineOptions): `"authoring"`, `"faithful"`, `"corpus"`, or `"equiv"`. - [TransformEngineOptions](https://texform.dev/api/js/transform-engine.md#TransformEngineOptions) - [TransformProfile](https://texform.dev/api/js/transform-engine.md#TransformProfile) ## `RuleLevel` Kind: typeAlias ```ts "authoring" | "faithful" | "corpus" | "equiv" ``` How aggressively a transform rule may rewrite, layered from least to most destructive of stylistic detail. A rule level is the first profile that accepts a rule's output as a suitable product; it is orthogonal to a rule's equivalence guarantee. --- # serialize Render a SyntaxNode to LaTeX text, and the output style options. https://texform.dev/api/js/serialize ## `serialize` Kind: function ```ts serialize(node: SyntaxNode, options: SerializeOptions | null | undefined): string ``` **Deprecated.** Prefer `Document.fromSyntax(node).toLatex(options)` or `document.toLatex(options)`. This remains exported as a compatibility helper for older snapshot call sites and takes the same options as `toLatex`. Render a [SyntaxNode](/api/js/parser#SyntaxNode) object to LaTeX text. `Error` nodes round-trip their captured snippet. An ordinary `Prime` emits `\prime` symbols, while a pure prime superscript serializes compactly as `f'` or `f''`. The serializer guarantees text idempotency. | Name | Type | Default | Description | | --- | --- | --- | --- | | `node` | SyntaxNode | — | - A [SyntaxNode](/api/js/parser#SyntaxNode) object, typically from [Document.toSyntax](/api/js/document#Document.toSyntax) or a stored snapshot. | | `options` | SerializeOptions | null | undefined | — | - A [SerializeOptions](/api/js/serialize#SerializeOptions) object, or omit/`null` for the default (spaced) style. | **Returns** `string` The serialized LaTeX string. ```ts import { Parser, serialize } from 'texform'; const result = new Parser().parse(String.raw`x^2`); if (!result.document) throw new Error('parse failed'); const syntax = result.document.toSyntax(); serialize(syntax); // 'x ^ { 2 }' serialize(syntax, { style: 'compact' }); // 'x^2' ``` - [SyntaxNode](https://texform.dev/api/js/parser.md#SyntaxNode) - [Document.toSyntax](https://texform.dev/api/js/document.md#Document.toSyntax) - [SerializeOptions](https://texform.dev/api/js/serialize.md#SerializeOptions) ## `SerializeOptions` Kind: interface Options for [Document.toLatex](/api/js/document#Document.toLatex), [Document.toTokenizedLatex](/api/js/document#Document.toTokenizedLatex), and [serialize](/api/js/serialize#serialize). Unknown keys and invalid values throw `TexformConfigError`. For a task-oriented walkthrough, see the Serialization guide. | Name | Type | Description | | --- | --- | --- | | `style` | Style | null | undefined | `"spaced"` (default when omitted or `null`) separates every token and is byte-stable across releases. `"compact"` keeps only required spaces and drops braces around single-token scripts; it preserves parsed structure, but its layout may improve across releases. | ```ts const doc = new Parser().parse(String.raw`\frac{a}{b}+x_i^2`).document; doc.toLatex(); // '\\frac { a } { b } + x _ { i } ^ { 2 }' doc.toLatex({ style: 'compact' }); // '\\frac{a}{b}+x_i^2' ``` - [Document.toLatex](https://texform.dev/api/js/document.md#Document.toLatex) - [Document.toTokenizedLatex](https://texform.dev/api/js/document.md#Document.toTokenizedLatex) - [serialize](https://texform.dev/api/js/serialize.md#serialize) ## `Style` Kind: typeAlias ```ts "spaced" | "compact" ``` Spaced output is byte-stable; compact preserves structure and text idempotency. --- # validateArgspec Validate an argspec string and inspect the parsed argument slots. https://texform.dev/api/js/validate-argspec ## `validateArgspec` Kind: function ```ts validateArgspec(spec: string): ValidateArgspecResult ``` Validate an xparse-style argspec string and report its parsed slots. Use it to self-check an argspec before injecting a custom command into a [Parser](/api/js/parser#Parser). Never throws on a malformed spec: the failure is reported through the `valid` / `error` fields of the result. For the notation semantics, see the Argspec guide. | Name | Type | Default | Description | | --- | --- | --- | --- | | `spec` | string | — | - The argspec string, such as `'o m'`, `'s m{}'`, or `` `d<(,)><[,]>` ``. | **Returns** `ValidateArgspecResult` A [ValidateArgspecResult](/api/js/validate-argspec#ValidateArgspecResult) describing validity and slots. ```ts import { validateArgspec } from 'texform'; validateArgspec('o m'); // { valid: true, error: null, argCount: 2, parsed: [ ... ] } validateArgspec('ABC123').valid; // false ``` - [Parser](https://texform.dev/api/js/parser.md#Parser) - [Argspec](https://texform.dev/docs/argspec.md) - [ValidateArgspecResult](https://texform.dev/api/js/validate-argspec.md#ValidateArgspecResult) ## `ValidateArgspecResult` Kind: interface The outcome of [validateArgspec](/api/js/validate-argspec#validateArgspec): whether the spec is well-formed and, when valid, its parsed slots. On success `valid` is `true` and `argCount` / `parsed` describe the slots; on failure `valid` is `false` and `error` carries the reason. | Name | Type | Description | | --- | --- | --- | | `valid` | boolean | Whether the specification parsed successfully. | | `error` | string | null | The error message when invalid, otherwise `null`. | | `argCount` | number | null | The number of argument slots when valid, otherwise `null`. | | `parsed` | ParsedArgSpecSlot[] | null | The per-slot breakdown when valid, otherwise `null`. | - [validateArgspec](https://texform.dev/api/js/validate-argspec.md#validateArgspec) ## `ParsedArgSpecSlot` Kind: interface One parsed argument slot of an argspec, as reported by [validateArgspec](/api/js/validate-argspec#validateArgspec) and by the parser's command/environment lookups. | Name | Type | Description | | --- | --- | --- | | `required` | boolean | Whether the argument is mandatory (`true`) or optional (`false`). | | `noLeadingSpace` | boolean | Whether leading space before the argument is disallowed. | | `nullable` | boolean | Whether an absent optional argument yields a null value. | | `kind` | ArgSpecKindInfo | The value kind the slot accepts. | | `form` | ArgSpecFormInfo | The syntactic form the slot takes. | - [validateArgspec](https://texform.dev/api/js/validate-argspec.md#validateArgspec) --- # Utilities List knowledge packages and the error classes. https://texform.dev/api/js/utilities The structured library errors derive from `TexformError` (which extends `Error`) and carry a `kind` discriminator, so one `instanceof TexformError` check catches all of them: - `TexformError` - `TexformParseError` — an operation needed a complete tree but parsing produced none. - `TexformEditError` — an invalid `Document` edit. - `TexformConfigError` — invalid construction or per-call configuration input. - `TexformTransformError` — a transform-engine failure. `Parser.parse` reports malformed LaTeX through its result object rather than throwing; invalid configuration still throws `TexformConfigError`. Passing `TransformEngine.transform` a document that `hasErrors()` throws `TexformTransformError` with `kind: "transform"`, not the base class. There is no target-counting helper here. Python's `count_targets`, which tallies command, environment, and character occurrences in a formula, is Python-only. To approximate it, parse the source and walk the `Document` with `findCommands` / `findEnvironments` or the `Node` navigation methods. ## `listPackages` Kind: function ```ts listPackages(): PackageInfo[] ``` List all built-in knowledge packages with record counts. The returned names are the identifiers accepted by the `packages` option of [KnowledgeBase](/api/js/knowledge-base#KnowledgeBase): `ams`, `base`, `bboldx`, `boldsymbol`, `braket`, `physics`, `textmacros`. **Returns** `PackageInfo[]` One [PackageInfo](/api/js/utilities#PackageInfo) per built-in package. ```ts import { listPackages } from 'texform'; listPackages(); // [ { name: 'ams', commands: 35, environments: 28 }, ... ] ``` - [KnowledgeBase](https://texform.dev/api/js/knowledge-base.md#KnowledgeBase) - [PackageInfo](https://texform.dev/api/js/utilities.md#PackageInfo) ## `PackageInfo` Kind: interface Summary of one built-in knowledge package, returned by [listPackages](/api/js/utilities#listPackages). | Name | Type | Description | | --- | --- | --- | | `name` | string | The package identifier, accepted by the `packages` option. | | `commands` | number | Number of command records in the package. | | `environments` | number | Number of environment records in the package. | - [listPackages](https://texform.dev/api/js/utilities.md#listPackages) ## `TexformError` Kind: exception Extends: Error Base class for every error the library throws. `TexformError` extends the built-in `Error`, so a single `catch` with an `instanceof TexformError` check catches all library errors. The [TexformError.kind](/api/js/utilities#TexformError.kind) discriminator identifies which subsystem raised it. - [TexformError.kind](https://texform.dev/api/js/utilities.md#TexformError.kind) - [TexformParseError](https://texform.dev/api/js/utilities.md#TexformParseError) - [TexformEditError](https://texform.dev/api/js/utilities.md#TexformEditError) - [TexformConfigError](https://texform.dev/api/js/utilities.md#TexformConfigError) - [TexformTransformError](https://texform.dev/api/js/utilities.md#TexformTransformError) #### `kind` Discriminator naming the subsystem that raised the error: `"parse"`, `"edit"`, `"config"`, `"transform"`, or `"internal"`. ## `TexformParseError` Kind: exception Extends: TexformError Thrown when source could not be parsed into a complete tree. [TransformEngine.normalize](/api/js/transform-engine#TransformEngine.normalize) throws this on input that cannot produce a complete tree, and `Document` construction and editing methods throw it for invalid source strings. Note that [Parser.parse](/api/js/parser#Parser.parse) itself never throws — it returns a [ParseResult](/api/js/parser#ParseResult) instead. - [TransformEngine.normalize](https://texform.dev/api/js/transform-engine.md#TransformEngine.normalize) - [Parser.parse](https://texform.dev/api/js/parser.md#Parser.parse) - [ParseResult](https://texform.dev/api/js/parser.md#ParseResult) #### `diagnostics` The diagnostics describing why parsing could not produce a complete tree. #### `document` The partial [Document](/api/js/document#Document) that was recovered, or `null` if none. ## `TexformEditError` Kind: exception Extends: TexformError Thrown by [Document](/api/js/document#Document) editing methods on an invalid edit: editing a read-only (error) tree, detaching or removing the root, an out-of-bounds index, or mixing nodes across documents. - [Document](https://texform.dev/api/js/document.md#Document) ## `TexformConfigError` Kind: exception Extends: TexformError Thrown on invalid construction or per-call configuration: unknown package or profile names, unknown or snake_case keys, arrays in object positions, and wrong scalar types. Messages include a camelCase field path. ## `TexformTransformError` Kind: exception Extends: TexformError Thrown on a transform-engine failure, such as an eliminated-form contract violation, or passing [TransformEngine.transform](/api/js/transform-engine#TransformEngine.transform) a document that [Document.hasErrors](/api/js/document#Document.hasErrors) or that is bound to a different [KnowledgeBase](/api/js/knowledge-base#KnowledgeBase) instance. - [TransformEngine.transform](https://texform.dev/api/js/transform-engine.md#TransformEngine.transform) - [Document.hasErrors](https://texform.dev/api/js/document.md#Document.hasErrors) - [KnowledgeBase](https://texform.dev/api/js/knowledge-base.md#KnowledgeBase) ## `TexformConformanceError` Kind: exception Extends: TexformEditError Thrown when a construction, edit, or syntax import violates the knowledge base or the document's structure. #### `path` The tree path where the violation was found. #### `rule` A stable machine-readable rule code. ## `listRules` Kind: function ```ts listRules(): RuleInfo[] ``` List all built-in rewrite rules, sorted by key. The keys are the identifiers accepted by the `disableRules` option of [TransformEngine](/api/js/transform-engine#TransformEngine) and reported in [RewriteReport](/api/js/transform-engine#RewriteReport) `rules`. A rule runs only when the engine's profile selects its level and the knowledge base enables one of its packages. **Returns** `RuleInfo[]` One [RuleInfo](/api/js/utilities#RuleInfo) per built-in rule. ```ts import { listRules } from 'texform'; listRules().find((rule) => rule.key === 'base/over-to-frac'); // { key: 'base/over-to-frac', level: 'authoring', fidelity: 'render', summary: '...', enabledByPackages: ['base'] } ``` - [TransformEngine](https://texform.dev/api/js/transform-engine.md#TransformEngine) - [RewriteReport](https://texform.dev/api/js/transform-engine.md#RewriteReport) - [RuleInfo](https://texform.dev/api/js/utilities.md#RuleInfo) ## `RuleInfo` Kind: interface Summary of one built-in rewrite rule, returned by [listRules](/api/js/utilities#listRules). | Name | Type | Description | | --- | --- | --- | | `key` | string | Stable rule key, such as `"base/over-to-frac"`, accepted by `disableRules`. | | `level` | RuleLevel | The first profile that accepts the rule's output. | | `fidelity` | RuleFidelity | Worst-case equivalence guarantee of the rule. | | `summary` | string | One-line description of what the rule does. | | `enabledByPackages` | string[] | Packages that make the rule loadable when any one of them is enabled. | - [listRules](https://texform.dev/api/js/utilities.md#listRules) ## `RuleFidelity` Kind: typeAlias ```ts "math" | "reading" | "render" ``` A rewrite rule's worst-case equivalence guarantee over its declared input domain, ordered least to most faithful: `math` (mathematical meaning is preserved, rendering may change), `reading` (notation content, reading order, and structural roles are preserved), `render` (rendering is equivalent under the reference renderer).