LoRA adapter

Overview#

A LoRA adapter is the only weight Ster ever trains. It is a pair of low-rank factors, A shaped [rank, in] and B shaped [out, rank], attached to one projection inside one decoder block. The update the projection receives is scale * x @ A^T @ B^T, written in that order so the widest thing ever allocated is [batch * sequence, rank] rather than the dense out × in matrix the factorisation exists to avoid.

Four objectives train them — `ster tune sft` on completions, `ster tune dpo` on preferences, `ster tune reward` under a Bradley-Terry head, and `ster tune grpo` by policy gradient; `ster tune merge` folds a finished one into the base weights; `ster tune evaluate` scores one without training it; ster tune inspect audits them (`ster tune inspect`); ster generate --adapter attaches them frozen (`ster generate`). The base checkpoint is mapped read-only and never registered as trainable, so nothing else can move.

Identity at initialisation#

A is drawn from Randn { mean: 0.0, stdev: 1/rank } and B starts at zeros. Because B is zero, the whole low-rank product is exactly zero before the first optimizer step, so a fresh adapter is the identity and training starts from the base model's own behaviour instead of from noise injected into every projection. It also means a first loss equal to the unadapted baseline is the correct reading, not a sign that nothing was wired up.

The scale a forward pass applies is alpha / rank. Dividing by the rank is what lets an operator raise the rank without also raising the effective size of every update, so --rank and --learning-rate stay independent knobs.

Which projections may be adapted#

Seven, named either the short way or the way a Hugging Face model card spells them:

Grouped-query attention makes key and value narrower than query, and the feed-forward block is wider than the residual stream, so the factors are not square and their shapes cannot be derived from hidden_size alone. Query outputs use attention_width, which need not equal the residual width. The output projection reads attention_output_width; this can differ from query width when value heads have another size. --targets query,value is the default, and --layers decides which blocks carry the chosen targets.

The two-file artifact#

An adapter is written as a pair of files that travel together, and the extension is replaced rather than appended so the two sort side by side in a directory listing:

• <name>.lora.safetensors — the factors, named layers.{layer}.{target}.a and layers.{layer}.{target}.b. That string is the artifact's contract, so any other tool can read the weights.
• <name>.lora.json — the identity sidecar: schema_version, product, model, model_revision, rank, alpha, targets, layers, hidden_size, and train, the report of the run that produced the adapter.

Splitting them is what lets Ster refuse an adapter trained against a different model before a single tensor is loaded, exactly as a steering artifact carries its own identity. The current schema_version is 1.

Refusals#

Validation is fail-closed, and it runs on save, on load, on inspection, and before generation:

• adapter artifact schema <N> is unsupported; this Ster build reads schema 1
• adapter artifact belongs to product "<PRODUCT>", not Ster
• adapter artifact declares rank 0, which stores nothing
• adapter artifact declares alpha <ALPHA>, which is not a positive finite number
• adapter artifact declares hidden size 0
• adapter artifact names no targets
• adapter artifact names no layers
• adapter artifact names target <TARGET> twice
• adapter artifact names layer <L> twice
• adapter artifact is missing tensor <NAME>
• adapter tensor <NAME> has <N> dimensions, expected 2
• adapter tensor <NAME> has <N> rows, expected <EXPECTED>
• adapter tensor <NAME> has <N> columns, expected <EXPECTED>
• adapter tensor <NAME> contains a non-finite value

Standalone inspection checks rank and each axis known from the sidecar's hidden_size; it does not assume query outputs have residual width. Attaching, exporting or merging an adapter also checks both factor axes against the checkpoint's actual projection widths. A mismatch is refused before use with adapter tensor <NAME> has shape <ACTUAL>, expected <EXPECTED> for the model projection. The CLI and Desktop operations use the same validation.

The real-checkpoint regression is bash tests/tuning/projection_widths.sh in the Ster repository. Set STER to the built binary, STER_TEST_MODEL to an absolute local published Qwen3 checkpoint with unequal query and residual widths, STER_TEST_EXAMPLES to supervised examples, and STER_TEST_LEARNING_RATE and STER_TEST_MAX_SEQUENCE explicitly. It trains and inspects query factors, attaches and exports them, then changes only its isolated checkpoint configuration to check named shape refusals from attachment, export and merge. Reports under target/real-tests/tuning/ retain the source revision, source patch, binary and checkpoint hashes, commands, exit statuses and outputs. The test never changes the supplied checkpoint.

• I/O adds failed to read adapter <PATH> for an absent artifact — the path asked for, not the sidecar derived from it — then adapter <PATH> has no sidecar at <SIDECAR>; the pair is written together and must travel together, failed to read adapter sidecar <PATH>, invalid adapter sidecar JSON in <PATH>, failed to read adapter weights <PATH>, and failed to read adapter tensor <NAME>.
• Handing the loader a steering artifact instead is refused by name, before serde sees it: direction.json is a steering artifact, not a LoRA adapter sidecar: it carries trait_name and vectors where an adapter sidecar carries rank and targets. The named path is the sidecar the loader derived from the weights, which for a .json argument is the file the operator gave. `SteeringArtifact::load` carries the mirror sentence for the same mistake made the other way.
• Writing adds failed to create <PARENT>, failed to write adapter weights <PATH>, and failed to write adapter sidecar <PATH>.
• Attaching one to a model adds the identity gates: adapter was trained for model "<ADAPTER_MODEL>", current model is "<MODEL>" and adapter width <WIDTH> does not match model width <WIDTH>.
• Building the spec that a run trains adds adapter rank 0 trains nothing; choose a rank of at least 1, adapter alpha <ALPHA> is not a positive finite number, adapter spec names no targets; choose at least one of query, key, value, output, gate, up, down, adapter spec names target <TARGET> twice, and adapter spec names layer <L>, but the model has <COUNT> layers.