ster vector optimize

Invocation and purpose#

ster vector optimize --model <MODEL> --output <OUTPUT> --holdout <FRACTION> [--pairs <PAIRS>] [--revision <REVISION>] [--device <DEVICE>] [--layers <LAYERS>] [--chat-template <auto|off>] [--precision <f32|f16|bf16>]

Evaluates every caa, pca, and logistic candidate at every selected layer on the last --holdout fraction of the pairs in file order, rounded to whole pairs. Highest accuracy wins, with margin as the tie-breaker; Ster then retrains that method and layer on the complete pair set. The score every candidate earned is published beside the winner, because those scores are the whole content of the decision and they cost nothing to carry: they were computed to make it.

Inputs and options#

Output and state#

Writes one single-layer steering artifact whose metadata.selection records the search that produced it — on the eight-pair toy set over all four layers at --holdout 0.2, chosen over 12 candidates on a 2-pair holdout, then refitted on all 8 pairs. The published direction is refitted on every pair, holdout included: the split existed to rank candidates, and once the ranking is done, throwing away the held-out evidence would be paying for the measurement twice. The scores stored in the artifact are therefore full-set training scores, not the holdout scores that selected it.

Stdout is the summary document `ster vector train` prints, plus one selection object carrying the evidence:

{ "selection": {
    "holdout": { "fit_pairs": 6, "holdout_pairs": 2 },
    "candidates": [
      { "layer": 2, "method": "caa",
        "holdout_accuracy": 1.0, "holdout_margin": 0.7814546823501587,
        "selected": false },
      { "layer": 2, "method": "pca",
        "holdout_accuracy": 0.5, "holdout_margin": -0.0392594039440155,
        "selected": false },
      { "layer": 3, "method": "caa",
        "holdout_accuracy": 1.0, "holdout_margin": 0.9493659734725952,
        "selected": true } ] } }

That excerpt keeps three of the twelve rows the eight-pair toy set produces over all four layers; the fields are exact. It was recorded when the split was a fixed 80/20, which --holdout 0.2 reproduces; a run today also reports "fraction": 0.2 inside holdout. holdout reports the two counts the fraction produced beside it, because a ratio is not what an operator needs to judge a ranking — two pairs is. candidates carries one row per layer and method, in the order the search walked them, layer outermost and caa, pca, logistic within each, so two runs over the same layers diff line for line. Exactly one row is "selected": true: the winner is flagged in place rather than moved to the front, for the same reason.

Every row also carries holdout_effect_size, recorded after that excerpt: Cohen's d of the held-out sides along the candidate's direction — the mean positive projection minus the mean negative one, over their spread pooled around each side's own mean — or null when the projections do not spread at all. It is the answer wisent's check-linearity gave, whether a trait lies along a line at a layer, as a number at every layer and method rather than a label cut at thresholds nobody stated. wisent's cluster-benchmarks is one vector train per benchmark grouped by `ster vector compare` --clusters; its geometry-search is `ster pairs merge` of the benchmarks then this command; its zwiad is this table with vector compare and `ster vector project`.

Stderr carries the template decision, one line per pair read, then fitting each candidate on 6 pairs and ranking on a 2-pair holdout. A one-pair holdout gets a second line: a one-pair holdout scores every candidate 0 or 1, so this ranking separates almost nothing; add pairs to make the choice mean something. That is a fact about the input rather than a defect in the command, so it is stated rather than refused.

Refusals#

• Leaving out --holdout is a usage error naming the flag, exit 2. A fraction outside (0, 1) is refused as the held-out fraction must be above zero and below one, and one that leaves either side without a pair as a held-out fraction of <F> over <N> contrastive pairs leaves <FIT> pairs to fit and <HELD> to rank on; both need at least one, before any activation is read.
• The pair loader refuses missing, invalid, empty, or partly blank pair files before optimization begins.
• Invalid layer selections, devices, unavailable GPU features, and unsupported model families or checkpoint settings are refused under the same contracts as `ster vector train`. Model resolution defines the supported families and settings.
• A candidate that produces a zero or non-finite direction is refused as training produced a zero or non-finite direction.
• --chat-template takes auto or off and nothing else; any other value is refused as unknown chat template mode "<VALUE>"; expected auto or off. A template the checkpoint publishes but MiniJinja cannot compile or render is refused at load, before a pair is read (chat template).