Overview#

A language model asked a question writes its answer one token at a time, and everything that makes generated text hard for software to depend on comes from that: the parsing, the refusals, the option that was never offered, the missing confidence. A decision needs none of it. When the options are fixed in advance, the whole answer is already in the distribution the model holds at one position. Ster shows the model the state, the question and the options under letters, ends the prompt where the answer would begin, and reads how much mass the next-token distribution puts on A, B, C and so on. That is one forward pass with no sampling: it cannot produce an option that was not offered, and it hands back a probability for every option that was.

The request and answer shapes are the ones TypeSafe's System One API uses for its hosted model Jev, so a workflow written against that API runs against a local open-weight checkpoint through `ster decide`. Ster makes no claim about matching that model's quality — TypeSafe has not published its design — it makes the mechanism available on a checkpoint you hold, where every number can be read back with --explain.

How the number is made honest#

The state is read once. Every question about one state begins with the same text, so the tokens all the renderings share go through the model once with the key-value cache on, and each rendering then runs only its own suffix — the question, the options, the answer prompt — on a copy of that cache. The shared run is found on the tokens rather than the text, because a tokenizer may merge across the point where two renderings diverge. On the recorded run this was 84 shared tokens and 380 across eight renderings.
Every option is shown under every letter. A model prefers some letters whatever they label — TinyLlama-1.1B-Chat answers B to almost anything — and one rendering cannot tell that preference from a reading of the content. A question with n options is rendered in all n cyclic orders, each rendering's letter distribution is normalized on its own, and the log-probabilities are averaged across the orders. Every option wears the favoured letter exactly once, so the favour lands on each of them equally and cancels; what the model read from the content survives in every order and adds up. --permutations caps the number of orders.
Each letter is read in every spelling. A vocabulary carries A bare, as SentencePiece's word-initial ▁A, and as byte-level BPE's ĠA; which one the model uses depends on whether the prompt ended in whitespace or a chat marker, so every spelling the vocabulary holds is scored and summed in the log domain.
The temperature can be fitted. `ster calibrate` divides the averaged log-scores by one number fitted on labelled decisions, which sharpens or flattens every distribution without changing which option wins, and reports a shuffled-state control so a model that never read the state is caught rather than calibrated.

What comes back#

A choice returns the winning option, a probability per option and a confidence of 1 − entropy / ln(n). A score returns the probability-weighted level, the per-level probabilities, the legend and the same confidence. A noul returns the probability of yes. usage counts only the tokens the model processed — the state once, then the suffixes — and output_tokens is always zero, because nothing is generated. --explain adds, per question, the probability the model gave each option in each order it was shown, which is how a flat answer is read: on TinyLlama the per-order rows show 0.78 wherever B sits and the average is uniform, while on SmolLM2-1.7B-Instruct returns wins in every order.