The knowledge base

Read this if you want to know where the parser's knowledge comes from — what each record holds, how an entry's argument structure is decided, how it is verified before shipping, and how to report one that is wrong.

The knowledge base is the layer that makes parsing more than string matching: for every name it supports, it records which package the name belongs to, which modes it is legal in, and what argument shape it takes. The parser consults it to decide how many arguments \frac consumes or whether \sqrt accepts an optional root. This page describes how those records are built and kept honest.

The four kinds of record#

A package specification holds four kinds of record, and each answers a different question for the parser.

Commands and environments are the entries you see when you ask a package what it contains, and they are the ones that carry an argspec — an xparse-style signature such as m m (two mandatory arguments) or o m (one optional, one mandatory). Characters and delimiters have no arguments to describe; they are what the symbol reference is generated from.

Records live in per-package specification files, one file per package, and are compiled into static tables when TeXForm is built, so the data the parser uses is the same data this site's references are generated from.

Which packages are loaded#

The built-in packages are base, ams, physics, textmacros, bboldx, boldsymbol, color, and braket. A parser or transform engine built without an explicit package list loads everything except physics, which stays opt-in because it redefines bra-ket commands that braket provides and standard commands such as \div, \Re, and \Im. Ask for physics by name when you need it, and list the other packages you want in the same call: naming packages replaces the default set rather than adding to it.

Where entries come from#

Entries are not transcribed by hand from prose documentation. They are built in stages:

• Package definitions are analyzed. The argument structure of each command is derived from how its package actually defines it, so the recorded shape reflects the real macro, not a documentation summary.
• Annotation is LLM-assisted. A language model proposes the argspec, allowed modes, and metadata for each entry, turning raw definitions into structured candidates.
• A human reviews the decisions. Proposed entries are reviewed and accepted, corrected, or rejected before they enter the shipped knowledge base. The machine proposes; a person decides.

How entries are verified#

Every entry's argspec is checked by rendering, not by inspection. From the argspec, test cases are generated with a one-factor-at-a-time strategy and rendered against three real engines — MathJax, KaTeX, and XeTeX. If the argument forms an entry claims do not render the way the spec predicts, that shows up as a renderer gap rather than passing silently. The full procedure, the three engines, and the meaning of the resulting full / partial / none statuses are described in renderer validation.

Frequency from corpus statistics#

Entries are weighted by how common they are. Command and environment frequencies come from counting occurrences across large real-world LaTeX corpora, so the catalog can distinguish a command that appears in millions of formulas from one that is rare. This is what lets renderer support rank gaps by how often a command actually shows up, rather than treating every entry as equally important.

How entries reach the runtime#

The reviewed specification files are compiled into TeXForm at build time and travel with the released Rust, Python, and JavaScript packages. There is no separate data download and no network lookup at parse time; the knowledge a build ships with is fixed for that release, and updates arrive with new releases.

That means the specification files are not a configuration surface — editing them requires rebuilding TeXForm from source. To teach a running parser about a command it does not know, register it yourself with a context item, as injecting custom commands shows. To narrow what a parser accepts, drop names from the knowledge it loads instead of adding to it.

Reporting an incorrect argspec#

If a command parses with the wrong argument structure — too many or too few arguments, a missing optional argument, a wrong mode — it is almost always a knowledge-base entry to fix rather than a parser bug. Open an issue on the TeXForm repository with the command name, the input you parsed, what you expected, and what TeXForm produced. A minimal reproducing formula is the most useful thing you can include.

Source map#

Paths are relative to the TeXForm repository root. They move as the code moves, so treat them as entry points rather than citations.

Related#