Command line

The texform command exposes the same engine as the Rust, Python, and JavaScript libraries to shell scripts and to tools that call TeXForm across a process boundary. It reads formulas from an argument or from stdin, and prints either plain text or one JSON value per formula.

Installation#

The texform command ships with the Python package, which has prebuilt wheels for Linux, macOS, and Windows. Install it as a standalone tool:

uv tool install texform    # or: pipx install texform

To run it once without installing, use uvx texform. In an environment that already uses the Python library, pip install texform installs the command too, also available as python -m texform.

With a Rust toolchain, cargo install texform-cli builds the same tool from source as a native binary. Both behave identically; the Python install adds about 10 ms of interpreter startup to each run, which matters only when calling the command once per formula. For many formulas, use --lines or texform serve instead.

Check that it runs:

texform --version

texform <version>

When the tool is built from a git checkout, the version line also carries the abbreviated commit hash and its date, as in texform <version> (<commit> <date>), followed by , dirty if the build inputs differed from that commit. A build installed from PyPI or crates.io has no git information, so only the version appears. The serve protocol reports the same identity in serverInfo.

Usage#

texform [--packages <a,b,...>] <COMMAND> [OPTIONS]

texform <COMMAND> --help lists every option of a command.

The one global option is --packages <a,b,...>, which picks the knowledge packages to load. It defaults to the same seven packages as the libraries (base, ams, braket, textmacros, bboldx, boldsymbol, and color); physics is opt-in, for the reasons given in Parsing. An unknown name is a usage error with exit status 2. The packages and argspec validate commands do not use it.

Formula input#

normalize, parse, and tokenize read formulas from one of three sources:

• The LATEX argument is one formula. Quote it with single quotes, such as '\sqrt{a+b}', so the shell preserves backslashes. Put -- before a formula that starts with -, for example texform parse -- '-x^2'; unknown options are rejected rather than read as formulas.
• Stdin without an argument is one formula: everything on stdin, minus one trailing newline (\n or \r\n). A formula with line breaks, such as an environment, stays whole.
• --lines makes every stdin line (ending in \n or \r\n) its own formula, and an empty line an empty formula. Lines are processed as they arrive, so the command can sit at the end of a pipe.

Input must be UTF-8.

Output and failures#

normalize prints one line of LaTeX per formula, and tokenize prints one line containing a token list. parse prints a syntax tree; with --lines, the trees are separated into blocks headed by <line N>. Add --json (with --lines for batches) to get one JSON value per input line instead.

A failing formula does not stop the run, and the rest of the input is still processed:

• In text mode, the error and diagnostics go to stderr. parse also shows any partial tree on stdout. Under --lines, a failed normalization or tokenization writes an empty placeholder line, so output lines still match input lines.
• In JSON mode, every formula produces one JSON value: successful normalization and parsing produce objects, successful tokenization produces an array, and any failure produces {"ok":false,"error":{...}}. Processing errors, including recovered panics, do not write to stderr; usage and configuration errors still do.
• When the downstream reader closes stdout (for example | head -n 5), processing stops immediately and the command exits successfully without reading the remaining stdin.

Diagnostics are colored only when stderr is a terminal and NO_COLOR is unset or empty.

Exit status#

Configuration with --config#

--config takes a JSON object, inline or as @FILE to read it from a file. Its values are layered over the defaults, and nested objects may be partial. Unknown keys and a zero rewrite.max_iterations are rejected before any formula is read, with exit status 2, and the error names the offending path, such as invalid --config: rewrite.enabld: unknown field ....

The normalize config has the same keys and resolves the same way as the keyword arguments of the Python binding, the camelCase options of the JavaScript binding, and the overrides object of the serve protocol's configure request. Transforms explains what each phase does.

texform normalize#

texform normalize --profile corpus 'a \over b'
texform normalize --profile corpus --style compact 'a \over b'
texform normalize --profile authoring --config '{"rewrite":{"enabled":false}}' --lines < formulas.txt
texform normalize --profile corpus --report --json 'a \over b'

--profile is required and takes one of authoring, faithful, corpus, or equiv; the profiles are the same ones the libraries use. The output is identical to what a TransformEngine returns for the same packages, profile, and config. --report adds the transform report to each result and requires --json. --style selects the output style, spaced (the default) or compact; it is a separate flag rather than a --config key. See Configure output style.

The example below feeds three formulas, one of them truncated. The failure leaves an empty line in place, 2>/dev/null hides its diagnostics, and the exit status is 1 because one formula failed. \dv is a physics command, so without that package it is an unknown command and passes through unchanged:

$ printf '%s\n' 'a \over b' '\frac{a' '\dv{f}{x}' | texform normalize --profile corpus --lines 2>/dev/null
\frac { a } { b }

\dv { f } { x }
$ echo $?
1

Loading physics through --packages turns the same line into the universal \frac form:

$ texform --packages base,ams,physics normalize --profile corpus '\dv{f}{x}'
\frac { \mathrm { d } f } { \mathrm { d } x }

texform parse#

Parses formulas without normalizing them. The default output is a compact syntax tree. --verbose prints the detailed tree as pretty-printed JSON and conflicts with --json, which emits a compact result object for scripts. Success means a complete document: a partial document containing error nodes still exits with status 1, but its tree remains visible.

$ texform parse '\sqrt{a+b}'
Root(Math) [
  Command(\sqrt, known=true) [
    Arg(None)
    Arg(Mandatory, no_leading_space=false):
      Group(Math, Implicit) [
        Chars("a+b")
      ]
  ]
]

The diagnostic source is named <argument>, <stdin>, or <line N> after where the formula came from. Under --lines, each tree is headed by <line N> and followed by a blank line, including empty formulas and failures. The tree is the same one Parsing describes.

texform tokenize#

Lists tokens from the spaced serialization of each formula, not from the original input; tokenize has no --style flag. With --profile, the formula is normalized before tokenization. Partial documents fail as in parse.

$ texform tokenize '\text{a b}'
[control_seq("\\text"), delim("{"), text("a b", mode=text), delim("}")]

The default output is a compact list of kind("text") entries. Human-readable kinds abbreviate character to char, control_sequence to control_seq, and delimiter to delim; text, raw, and error keep their names, and JSON keeps the full kind names. Math mode is implicit, so only non-math tokens show their mode. Text uses JSON string quoting to preserve spaces and escape newlines, quotes, and backslashes. An empty formula prints []. Each formula occupies one output line.

--json emits an array of token objects directly, with no ok or latex wrapper. Each object has text, kind, mode, start_byte, and end_byte. Under --lines, each successful input line produces one array on one output line, and a failure produces the standard error object, so output lines keep matching input lines.

Offsets refer to the spaced serialization (the normalized serialization when --profile is given), not to the input string. Spaces between tokens belong to that serialization, so joining token texts with spaces is not a way to reconstruct it, particularly in text mode.

texform info#

texform info '\frac'
texform info --env align
texform info --mode text '\textbf'

Looks up a name in the knowledge base of the loaded packages, in math mode unless --mode text is given.

• A control sequence such as \frac shows its command record and, when one exists, its character record. Character commands such as \alpha have both: the character record gives the Unicode value, and the command record shows that the name parses as a command without arguments. When both records are shown, a note identifies the command record as the one used for parsing.
• --env NAME shows an environment record.
• Anything that is neither a control sequence nor used with --env is a usage error. Characters are looked up by their control-sequence name; there is no lookup by Unicode character.

When nothing is found, the exit status is 1.

$ texform info '\alpha'
command:  \alpha
kind:     prefix
mode:     math
packages: base
argspec:  (no arguments)

character: \alpha
unicode:   α
mode:      math
variant:   italic
package:   base

Parsing uses the command record above. The character record is separate metadata.

texform packages#

Lists the built-in knowledge packages with the number of commands and environments in each. The catalog includes physics, even though it is not loaded by default.

$ texform packages
ams         35 commands, 28 environments
base        270 commands, 7 environments
bboldx      12 commands, 0 environments
boldsymbol  1 command, 0 environments
braket      11 commands, 0 environments
color       5 commands, 0 environments
physics     182 commands, 1 environment
textmacros  78 commands, 0 environments

texform argspec validate#

Checks an xparse-style argument specification and describes each argument slot. An invalid specification is reported on stderr with exit status 1. The syntax is described in ArgSpec.

$ texform argspec validate 'm O{default} m'
valid: 3 arguments
  1. required, math content
  2. optional, math content
  3. required, math content

JSON output#

--json output uses the same data structures as the Python and JavaScript bindings, so field names and values match the Python binding exactly and the JavaScript binding up to camelCase. Member order is not significant.

• error.kind is parse (the formula did not parse into a complete tree), transform (a normalization step failed), or internal (an internal error, including a panic inside TeXForm, which fails only that formula).
• error.diagnostics lists parse diagnostics for parse failures and is empty otherwise. Each diagnostic has kind, message, span (start and end byte offsets into the formula), expected, found, and contexts. Failures carry syntax only from parse, and only when a partial document exists.
• syntax is the serialized syntax tree that Document.to_syntax() returns in Python and toSyntax() in JavaScript.
• report is the transform report. Its fields are diagnostic and carry no compatibility promise.
• Each token has kind of control_sequence, character, delimiter, text, raw, or error, and mode of math or text. The offsets are UTF-8 byte offsets, like the spans the libraries report.
• The info records are the same results the bindings return from their command, character, and environment lookups.

$ texform normalize --profile corpus --json '\frac{a'
{"ok":false,"error":{"kind":"parse","message":"parse produced an incomplete document","diagnostics":[{"kind":"argument-validation","message":"unclosed brace argument","span":{"start":5,"end":7},"expected":[],"found":null,"contexts":[{"label":"command argument","span":{"start":5,"end":7}}]}]}}
$ texform tokenize --json 'x^2'
[{"text":"x","start_byte":0,"end_byte":1,"kind":"character","mode":"math"},{"text":"^","start_byte":2,"end_byte":3,"kind":"character","mode":"math"},{"text":"{","start_byte":4,"end_byte":5,"kind":"delimiter","mode":"math"},{"text":"2","start_byte":6,"end_byte":7,"kind":"character","mode":"math"},{"text":"}","start_byte":8,"end_byte":9,"kind":"delimiter","mode":"math"}]

Serve the normalizer protocol#

texform serve is a long-running process that normalizes formulas on request. It speaks JSON-RPC 2.0 over stdin and stdout, so another program can start it once and send many formulas without paying process start-up and rule-plan compilation for each one. This is the command-line counterpart of reusing one engine. The protocol is independent of TeXForm: any normalizer can implement it, and a client written against it can measure any implementation.

texform serve [--packages <a,b,...>]

--packages only sets the default for configure requests that omit packages.

Transport#

• Each message is one line of UTF-8 JSON terminated by \n. Messages must not contain raw newlines; JSON string escapes such as \n are fine.
• The server reads requests from stdin and writes only responses to stdout, one per line, flushing after each. stderr carries human-readable logs that clients must not parse.
• Requests are processed one at a time in arrival order, and responses are written in the same order. Clients may pipeline requests or wait for each response.
• A request without an id member is a notification: it is executed, but no response is written, even on failure. id may be a string, a number, or null.
• Batch requests (JSON arrays) are not supported and receive a single -32600 error. Blank lines are ignored.
• When stdin reaches end of file, the server finishes the requests it has read and exits with status 0, whether or not shutdown was sent.

A session looks like this, where → is client to server and ← is server to client, with some lines abbreviated:

→ {"jsonrpc":"2.0","id":0,"method":"initialize","params":{}}
← {"jsonrpc":"2.0","id":0,"result":{"protocolVersion":1,"serverInfo":{"name":"texform","version":"...","commit":"...","commitDate":"...","dirty":false}}}
→ {"jsonrpc":"2.0","id":1,"method":"configure","params":{"id":"authoring-no-rewrite","config":{"profile":"authoring","overrides":{"rewrite":{"enabled":false}}}}}
← {"jsonrpc":"2.0","id":1,"result":{"resolved":{"profile":"authoring","packages":["ams","base","..."],"rewrite":{"enabled":false,"max_iterations":100},"...":"..."}}}
→ {"jsonrpc":"2.0","id":2,"method":"normalize","params":{"config":"authoring-no-rewrite","latex":"\\frac12","timing":true}}
← {"jsonrpc":"2.0","id":2,"result":{"output":"\\frac { 1 } { 2 }","timing":{"parse_ns":41250,"transform_ns":35980,"serialize_ns":2410}}}
→ {"jsonrpc":"2.0","id":3,"method":"normalize","params":{"config":"authoring-no-rewrite","latex":"\\frac{a"}}
← {"jsonrpc":"2.0","id":3,"error":{"code":1,"message":"parse produced an incomplete document","data":{"kind":"parse","diagnostics":[...]}}}
→ {"jsonrpc":"2.0","id":4,"method":"shutdown","params":{}}
← {"jsonrpc":"2.0","id":4,"result":null}

Methods#

params must be an object; initialize and shutdown also accept omitted params. Unknown members of params itself are ignored, while unknown members of config and overrides are rejected (see Versioning).

initialize#

Must be the first request. Any other method received earlier fails with -32002. Repeating initialize is harmless and returns the same result. serverInfo identifies the build, like texform --version:

configure#

Builds a normalizer from config and stores it under id for later normalize requests. Configuring an existing id replaces its entry; a failed configure leaves the previous entry unchanged.

• profile is one of authoring, faithful, corpus, or equiv. It selects the rule set and the default config.
• style is the output style of normalize results, spaced (the default) or compact. It sits beside profile, not inside overrides.
• packages lists the knowledge packages to load. When omitted, the server's --packages value applies, and without that, the same seven default packages as the libraries (physics is opt-in). Clients that need a fixed selection should specify it explicitly and record resolved.
• overrides are config values layered over the profile defaults. They accept any subset of the keys in resolved other than style, profile, and packages, with the same nesting; nested objects may also be partial.

resolved is the complete effective config, suitable for recording alongside results:

{
  "style": "spaced",
  "profile": "authoring",
  "packages": ["ams", "base", "bboldx", "boldsymbol", "braket", "color", "textmacros"],
  "reject_unknown": false,
  "abort_on_error": false,
  "max_group_depth": 128,
  "lower_attributes": { "enabled": true },
  "rewrite": { "enabled": false, "max_iterations": 100 },
  "finalize_ast": { "enabled": true },
  "flatten_groups": { "enabled": true, "preserve_rendered_spacing": true }
}

packages lists the loaded set without duplicates in catalog order, regardless of the order requested. Key order within resolved is not significant. An unknown style, profile, or package, or an invalid override (including rewrite.max_iterations = 0, even when rewriting is disabled) fails with -32602 before the configuration is registered.

normalize#

Normalizes latex with the configuration stored under config. The result is identical to what a TransformEngine returns for the same input and effective config. A formula that fails to normalize is a normal outcome, reported as error code 1 (see Errors). An unknown config id fails with -32602.

With report: true, the response also carries the transform report, including warnings for unknown input commands and environments. The report uses snake_case, as in normalize --json --report. The plain response omits report, and requesting it does not change output or error handling. With timing: true, the response also carries per-stage durations (see Timing).

shutdown#

Returns null. Every later request fails with -32600, and the server then only waits for end of file on stdin.

Errors#

Error responses carry a JSON-RPC error object {"code","message","data"?}. There are two layers: protocol errors mean the request itself was invalid, while code 1 is a per-formula result that callers should treat as data.

• For -32602, message is "Invalid params" and data.message explains the problem, including the path of an offending field such as config.overrides.rewrite.enabld.
• For code 1, message describes the failure and data.kind classifies it as parse (the formula did not parse into a complete tree), transform (a normalization step failed), or internal (an internal error, including a panic inside the server; the server stays usable after a panic).
• data.diagnostics lists parse diagnostics for parse failures and is empty otherwise. Each diagnostic has kind (a stable kebab-case identifier or null), message, span (start and end byte offsets into latex), expected, found, and contexts.
• The server applies no timeouts. A client that needs one should kill and restart the process.

Timing#

With timing: true, a successful normalize result contains timing with three members, each an integer number of nanoseconds:

• Timings cover only the server's internal work, not JSON decoding, encoding, or I/O.
• A code 1 failure carries timing in error.data.timing with only the stages that ran: a parse failure has only parse_ns, and a transform failure has parse_ns and transform_ns. Failures caused by a panic carry no timing.
• Requesting timing never changes output or the error classification.
• Whether a server supports timing is decided from successful responses; an implementation without timing support ignores the member.

Versioning#

• protocolVersion increases only when method semantics or required fields change incompatibly. New optional fields and new methods keep the version.
• Clients must refuse to continue with a protocolVersion they do not recognize.
• Servers ignore unknown members of params, so clients can send newer optional fields to older servers. Unknown members of config and overrides are rejected instead, so a configuration an older server cannot honor fails at configure rather than silently producing different results.

Next steps#