ster tune sft

Invocation and purpose#

ster tune sft --model <MODEL> --examples <EXAMPLES> --output <OUTPUT> --rank <RANK> --alpha <ALPHA> --epochs <EPOCHS> --learning-rate <LEARNING_RATE> --accumulation <ACCUMULATION> --max-sequence <MAX_SEQUENCE> --batch-size <BATCH_SIZE> --seed <SEED> [--revision <REVISION>] [--device <DEVICE>] [--targets <TARGETS>] [--layers <LAYERS>] [--warmup-steps <WARMUP_STEPS>] [--chat-template <auto|off>] [--precision <f32|f16|bf16>]

Supervised fine-tuning of a local checkpoint: for every example, the prompt and the completion are tokenized into one sequence, the model predicts the completion, and the gradient of that error reaches the adapters and nothing else. The base weights are mapped read-only and never registered as trainable, so “only the adapters train” is structural rather than conventional — the optimizer is handed the adapter tensors and there is nothing else in the map it could reach. A fresh adapter is the identity, so the run starts from the base model's own behaviour (LoRA adapter). The gradient exists at all because the decoder can run a differentiable pass.

--examples is a JSON document shaped {"examples": [{"prompt": "…", "completion": "…"}]}, with an optional name. Eight examples written entirely in the toy checkpoint's ~60-word vocabulary are checked in as `examples/examples.json`, so the command runs offline exactly as `examples/pairs.json` makes ster vector train run offline.

ster tune sft --model toy-llama --examples examples.json \
  --output calm.lora.safetensors --rank 8 --targets query,value \
  --epochs 3 --learning-rate 5e-3 --accumulation 4

On the four-layer, hidden-width-64 toy checkpoint that run reports 16 trainable tensors and 7168 trainable parameters, a first loss of 4.1335, a final loss of 4.0327, and a mean final-epoch loss of 4.0182. The same command run twice writes a byte-identical adapter, because --seed seeds the draw that fills every A factor as well as the per-epoch example order.

Inputs and options#

--batch-size examples go through each forward pass and --accumulation of those forwards are folded into one optimizer step from their scaled losses, so a step sees up to batch-size * accumulation examples. A batched forward right-pads its rows and masks every padded key out of every real query; at a batch size of one, the stride is one row per forward, which is what reproduces every run recorded before batching existed byte for byte. Batching changes how many sequences share a kernel launch, not what is optimized.

The loss is next-token cross-entropy over the completion tokens only. For a joined sequence of n tokens whose completion begins at boundary, the scored logits are narrow(1, boundary - 1, n - boundary) against the targets ids[boundary..]: the distribution that predicts a token sits one position to its left, and the prompt is never a target. The learning rate ramps linearly over --warmup-steps steps and then decays on a cosine to zero, the schedule of SGDR (Loshchilov and Hutter, 2017) and of Hugging Face's get_cosine_schedule_with_warmup; progress counts the steps taken, so the last step still learns.

Training loads no gateway, spends no quota, and writes nothing but the adapter pair.

Output and state#

Writes two files that travel together: <name>.lora.safetensors with the factors named layers.{layer}.{target}.a and layers.{layer}.{target}.b, and <name>.lora.json beside it with the identity and the training report (LoRA adapter). The checkpoint itself is untouched.

Stdout is a pretty JSON document with path and report. The report carries examples, trained_examples, skipped_long, epochs, steps, trainable_tensors, trainable_parameters, first_loss, final_loss, mean_final_epoch_loss, rank, alpha, targets, layers, learning_rate, accumulation, batch, chat_template, and precision, and the same report is recorded inside the sidecar, so a trained adapter always carries the run that produced it.

Stderr gets one line naming the trainable tensor count, the parameter count, the rank, and how many layers carry adapters, then one line per optimizer step with the epoch, the step out of the total, the examples seen, and that step's mean loss, then one mean-loss line per epoch. Every example over the limit gets its own line, for example skipping example 3: 17 tokens exceed the 3 token limit.

Refusals#

• Every training number is the caller's: leaving out --rank, --alpha, --epochs, --learning-rate, --accumulation, --max-sequence, --batch-size or --seed is a usage error that names the flag, exit 2, before anything loads (error: the following required arguments were not provided: --learning-rate <LEARNING_RATE>).
• --targets is parsed before anything loads, so an unreadable projection name is refused first: unknown adapter target "attn"; expected one of query, key, value, output, gate, up, down.
• The model then loads through the shared path, so its refusals apply here — unknown devices, GPU backends this binary was built without, and model architecture <TYPE> is unsupported by this Ster build; use a Hugging Face checkpoint whose model_type is one of <FAMILIES> (model resolution).
• The adapter spec is then checked against the model's real layer count, as the adapters are built: adapter rank 0 trains nothing; choose a rank of at least 1, adapter alpha <ALPHA> is not a positive finite number, adapter spec names no targets; choose at least one of query, key, value, output, gate, up, down, adapter spec names target <TARGET> twice, and adapter spec names layer <L>, but the model has <COUNT> layers.
• The example set is loaded and validated like a pair set: failed to read example set <PATH>, invalid example set JSON in <PATH>, example set <PATH> contains no examples, and example set <PATH> contains an empty prompt or completion.
• The training options are checked before the first gradient: supervised fine-tuning requires at least one epoch, supervised fine-tuning requires an accumulation of at least one example, supervised fine-tuning requires a finite learning rate above zero, and max_sequence must be at least two tokens, so that one token can predict another.
• Encoding refuses an example nothing can be learned from: training example has an empty completion, training example completion produced no tokens, and training example encodes to fewer than two tokens, so there is nothing to predict.
• Skipping a long example is not a refusal, but skipping every example is: every example is longer than the sequence limit, so there is nothing to train on.
• The artifact is validated before it reaches disk, so a file Ster would refuse to read is never written; the write itself reports failed to create <PARENT>, failed to write adapter weights <PATH>, or failed to write adapter sidecar <PATH>.