ster extract
Export final-token hidden representations for arbitrary prompts at selected model layers.
Invocation and purpose#
ster extract --model <MODEL> --input <INPUT> --output <OUTPUT> [--revision <REVISION>] [--device <DEVICE>] [--layers <LAYERS>] [--chat-template <auto|off>] [--precision <f32|f16|bf16>]Loads the model once and records each prompt's final-token residual state after every selected transformer block.
Inputs and options#
| Input | Required | Meaning |
|---|---|---|
--model <MODEL> | yes | Hugging Face model id or local model directory. |
--input <INPUT> | yes | JSON document shaped as {"prompts":["..."]} with at least one non-empty prompt. |
--output <OUTPUT> | yes | Destination activation-artifact JSON file. |
--revision <REVISION> | no | Immutable Hugging Face revision; Hub resolution defaults to main. |
--device <DEVICE> | no | cpu (default), metal, or cuda. |
--layers <LAYERS> | no | all (default), comma-separated indices, or half-open ranges. |
--chat-template <auto|off> | no | auto by default. auto reads every prompt through the model's own chat template when the checkpoint publishes one, off reads it as raw text (chat template). The exported activations are the states the model reached; this names what it was reading when it reached them, and the two are different states for the same prompt. |
--precision <f32|f16|bf16> | no | f32 by default. The dtype the base weights are mapped at: f32, f16, or bf16. bf16 is refused on cpu at load with `bf16 has no CPU matmul kernel in this Candle build; use --precision f16 for half precision on cpu, or --device metal for bf16` (precision). The exported activations are F32 either way; this names the width they were computed in. |
Output and state#
Writes a compact schema-1 activation artifact with product and model identity, hidden width, and one prompt record whose layers map contains each requested activation vector. Stdout is the output path; extracting prompt <N>/<TOTAL> progress goes to stderr.
Refusals#
- Unreadable input is refused as
failed to read <PATH>and malformed JSON asinvalid prompt JSON in <PATH>. - An empty prompt array is refused exactly as
prompt set contains no prompts; any blank prompt is refused asprompt must not be empty. - Invalid or out-of-range layer selections, unknown/unavailable devices, and unsupported model families or checkpoint settings are refused before the file is written. Model resolution defines the supported families and settings.
- A filesystem write failure is reported as
failed to write <OUTPUT>. --chat-templatetakesautooroffand nothing else; any other value is refused asunknown chat template mode "<VALUE>"; expected auto or off. A template the checkpoint publishes but MiniJinja cannot compile or render is refused at load, before a pair is read (chat template).