Invocation and purpose#

text

ster parity --model <MODEL> --input <INPUT> [--revision <REVISION>] [--device <DEVICE>] [--precision <f32|f16|bf16>]

Loads the model once, runs every record's token ids through it exactly as given — no tokenizer, no chat template — and compares the residual stream after the final norm at each position with the reference states recorded for those positions. It is how a family Ster implements without a trained checkpoint small enough to run is checked against the implementation it follows: the reference captures published beside tiny random-weight fixtures (Transformers' own forward pass, saved as safetensors) are read as they are.

Inputs and options#

Input	Required	Meaning
`--model <MODEL>`	yes	Hugging Face model id or local model directory: the checkpoint the reference states were recorded from.
`--input <INPUT>`	yes	JSON document `{"records": [{"tokenIds": [...] or "tokenIdsFile": "ids.json", "hidden": "states.safetensors", "tensor": "hidden_states"}]}`. Paths are relative to the input file; `tensor` defaults to `hidden_states`.
`--revision <REVISION>`	no	Immutable Hugging Face revision; Hub resolution defaults to `main`.
`--device <DEVICE>`	no	`cpu` (default), `metal` or `cuda`.
`--precision <f32\|f16\|bf16>`	no	Dtype the base weights are mapped at; the states are compared in F32 either way.

Each reference tensor holds the states after the final norm for the first P positions of its record, [P, hidden] or [1, P, hidden]; P may be fewer than the record's tokens, as a capture that keeps only the positions that predict a next token does.

What it reports#

One JSON document (or path: value lines under --text): the model and its resolved revision, then for every record and for all records together the number of compared positions and how far Ster's states lie from the reference ones.

Field	Meaning
`relativeError`	`‖ster − reference‖ / ‖reference‖` over every compared position.
`worstPositionRelativeError`	The same ratio for the single worst position.
`largestDifference`	The largest absolute difference of any one component.
`meanCosine`, `smallestCosine`	The mean and the smallest cosine between a position's two states.

A reference computed at a narrower precision than Ster's sets the floor these reach: a BF16 capture compared with Ster's F32 forward pass differs by rounding alone. On the published Transformers captures of the tiny random fixtures, the relative errors Ster reached are 0.47 % for MiniMax-M2 (malaiwah/minimax-m2-tiny-random-bf16), 1.6 % for Kimi-K2.5, 1.7 % for Kimi-K3, 2.0 % for GLM-5-Next and 2.8 % for DeepSeek-V4, whose indexer selection and hyper-connection streams carry the rounding further; every mean cosine is above 0.999. Before Ster normed MiniMax-M2's query and key whenever use_qk_norm is absent and read partial_rotary_factor from Transformers 5's rope_parameters, the same run reported 29 %.

text

$ ster parity --model malaiwah/minimax-m2-tiny-random-bf16 --input parity.json
{
  "model": "malaiwah/minimax-m2-tiny-random-bf16",
  "modelRevision": "a8eff3b80ee08b3ae340b30e673894269543288e",
  "overall": {
    "largestDifference": 0.023862123489379883,
    "meanCosine": 0.9999889109835776,
    "positions": 126,
    "relativeError": 0.004735244454055033,
    "smallestCosine": 0.9999759173306583,
    "worstPositionRelativeError": 0.006944841616870192
  },
  "records": [ … one entry per record, with its "hidden" file and "tokens" … ]
}

Refusals#

An input that is missing or not the document above: failed to read <path> or invalid parity JSON in <path>.
<input> lists no records to compare.
A record naming both or neither of tokenIds and tokenIdsFile, or holding no ids.
record <i> holds token id <id>, past the model's <n> tokens.
<file> holds no tensor "<name>"; it holds [...], naming the tensors the file does hold.
A reference tensor that is not [positions, hidden], holds more positions than the record has tokens, or is a different width than the model's hidden state.

The desktop app runs the same comparison as the parity request: {"model": …, "input": …, "precision": …}, answered with the same document.