Architecture

Overview#

Ster is one native Rust process with a thin CLI or desktop request front end over the same model, representation, and workflow code. It has no daemon, database, remote control plane, Python layer, or second inference implementation. Each run is its own process and ends when its work ends; nothing of Ster stays resident between two commands or two desktop requests.

CLI (main.rs, cli/) ────────┐
                            ├─ artifact.rs ─ pair/artifact JSON + validation
request/ (stdin → NDJSON) ──┤
                            ├─ workflow/ ─ capture/train/select/evaluate/export
                            ├─ pairs/ ─ pair inspection + synthesis
                            │      └─ dedupe / refusal / diversity
                            ├─ representation.rs ─ CAA/PCA/logistic + measures
                            └─ runtime/ ─ resolution/tokenization/generation
                                       │
                                       └─ model/ ─ native Candle Llama loop

Boundaries by module#

Front ends

main.rs and cli/ parse the subcommands and their pairs and tune groups. They load inputs, call library workflows, and print results; progress stays on stderr. request/ runs the same workflows for a desktop app: ster request <operation> reads one JSON body on stdin, validates its fields before anything runs, streams progress as NDJSON log events, and ends with one result event and the matching exit status. It does not shell out to the CLI: each operation calls the same Rust functions as the corresponding command. One process runs one operation, so the process-global progress sink never has two writers.

Data and representation

artifact.rs owns the pair-set and steering-artifact schemas and their fail-closed validation. representation.rs is model-independent numerical code: it accepts F32 activation rows, trains one normalized direction, and scores pair ordering. It never sees tokens, model files, or HTTP.

Workflow orchestration

workflow.rs captures both sides of each pair through Runtime, groups rows by layer, trains and scores vectors, performs the 80/20 optimization split and builds the candidate table it publishes, and exports activation JSON. It owns the artifact summary train, optimize and inspect all print, the progress messages, and the report wire shapes. It does not own model kernels.

Pair authoring

pairs.rs owns the pair-set audit and the synthesis loop, and it is the single implementation behind both the ster pairs subcommands and the pairs/* request operations. It calls Runtime for generation and nothing lower. dedupe.rs (SimHash plus banded LSH), refusal.rs (lexical refusal scoring), and diversity.rs (distinct-n, Jaccard, SimHash spread) are pure text modules: no model, no files, no network. Inspection uses only those three, which is why it needs no runtime at all.

Runtime and model

tune.rs is a module root rather than an objective. It owns what more than one trainer needs — the preflight checks, the example- and pair-set encoders, the log-probability readers, the softplus, and the learning-rate schedule — and each objective is a sibling file under tune/ owning only its own loss, options and report: sft.rs, dpo.rs, reward.rs, grpo.rs, plus the two utilities merge.rs and evaluate.rs. lora.rs owns the adapter factors, the artifact and its kind, and both halves of the tensor-naming contract.

runtime.rs resolves local or Hub files, selects a device, parses the tokenizer/config, maps Safetensors, gates artifacts against the loaded model, and runs tokenization, sampling, EOS/context stopping, and KV-cache lifecycle.

model.rs owns the Llama decoder loop: embeddings; RMS-normalized attention; rotary position embedding; grouped-query KV repetition; causal masking; gated MLP; residual additions; final norm; and LM head. Two product-specific hooks sit after each decoder block: copy the final-token residual state, then optionally add the layer's steering vector. Owning this loop is what makes the read and intervention points exact. It also lets the same loop run a second way: Pass::Differentiable replaces Candle's fused rotary_emb::rope, ops::softmax_last_dim, and ops::rms_norm — each of which ends in an apply_op*_no_bwd call and records no backward node — with composed equivalents at exactly those three call sites, which is what lets a loss reach an adapter at all (differentiable pass). Inference keeps the fused kernels, and the swap is chosen per call.

Data flow#

Train or optimize

pairs.json → validate → tokenize each side → model forward/capture
           → rows by layer → train normalized direction → evaluate
           → validate artifact → write *.ster.json + stdout summary

optimize captures once, evaluates every method/layer candidate on its ordered holdout, retrains the winner on all rows, writes one vector, and returns every candidate's holdout score beside it.

Generate

prompt → tokenize → optional artifact load/validate/model gate
       → steering plan on device → prompt forward + cached token forwards
       → sample/argmax until limit/EOS/context → decode generated suffix

No checkpoint weight is changed. The steering plan and KV cache die with the call.

Desktop request

stdin JSON body → parse/defaults/required fields (refusal: status 2)
                → shared workflow → progress log events on stdout
                → one result event → exit with the result's status

There is no listener and no queue. Two requests are two processes, each loading its own model; a reader that closes stdout ends its request at the next event.

Persistent and external state#

Ster writes only paths explicitly requested for steering artifacts, activation exports, or adapter pairs. inspect, evaluate, and generate do not mutate them. For a Hub id, hf-hub maintains its normal disk cache and optional token file; for a local directory, model resolution is offline. Ster has no application configuration file, telemetry, usage ledger, or automatic upload.

Trust boundaries#

• Model files and JSON inputs are untrusted. Parse, architecture, width, finiteness, model identity, and layer gates stop before intervention.
• Safetensors are memory-mapped with Candle's unsafe loader. Only load checkpoints from a source you trust and keep file permissions local.
• `ster request` trusts its caller. Paths in a request body are resolved on the filesystem of the process, relative to its working directory, with the permissions of the user who started it; a desktop app runs it as that user on its own Mac. No port is opened, so nothing outside that user's processes can submit work.
• Hub access may send the model id/revision and token to Hugging Face. Use a local directory for a no-network boundary.
• Generated text is data, not an authorization decision. Ster changes representations; it does not enforce policy or certify semantic outcomes.

Product ownership#

Ster owns local open-weight representation capture, direction learning, evaluation, residual steering, and LoRA adapter training under four objectives — supervised, preference, reward modelling, and policy gradient — plus merging, held-out scoring, and artifact inspection. Brama owns hosted model routing. Stado owns release delivery, and performs it: it installs the built binary from this repository's .wisent-release.json to ~/.stado/bin/ster (install path). Fleet placement and credentials are owned by nobody on Ster's behalf — Ster declines both rather than delegating them. The compute registry declares no Ster placement profile, no Ster OS unit and no Ster workload, and Skarbiec has no record of Ster in its source or its documentation; a run happens on the machine an operator starts it on. Ster calls none of these products.

Concurrency and resource model#

One Runtime owns one loaded model. Capturing one pair performs two full prompt forwards; capturing more layers increases copied activation memory but not the number of forwards. Training allocations scale with pair count × hidden width per selected layer. Generation uses a KV cache up to the context limit. Artifacts store selected_layer_count × hidden_size F32 numbers as JSON.

One process runs one command or one desktop request, so all work inside it is serial and the process-global progress sink has one writer. Library callers using set_progress_sink from several threads must provide that serialization themselves.