Ster
Ster/Docs

Architecture

A decoder loop built around exact intervention points.

Generic generation APIs do not expose the residual-stream operations Ster needs, so Ster owns its Llama decoder loop.

Native path

  • Memory-mapped Safetensors weights.
  • Tokenizer-driven prompt encoding.
  • Causal attention with KV caching.
  • Final-token residual capture after any selected block.
  • Direction addition before the next block.
  • CPU, Metal, or CUDA Candle kernels.

Model contract

The current runtime accepts config.json files declaring model_type llama, a tokenizer.json, and one or more Safetensors weight files. Other model types stop before weight loading.

Source

The implementation contract lives with the runtime at github.com/wisent-ai/ster. This website renders public product documentation and does not become a second source of implementation truth.