Model resolution

Overview#

Model resolution turns the --model, optional --revision, and --device inputs into one F32 Candle Runtime, from a local directory or the Hub. Ster implements one rotary, grouped-query decoder and reads what each family adds from the checkpoint's config.json: llama; mistral (sliding window on every layer when sliding_window is set); qwen2 (query/key/value bias, sliding window from max_window_layers when use_sliding_window); qwen3 (per-head query and key norms, stated head_dim); phi (Phi-2: LayerNorm with bias, a parallel block, biased projections named dense, fc1 and fc2, a plain GELU feed-forward, a biased head and final_layernorm); phi3 (fused qkv_proj and gate_up_proj, split at load and merged back row by row; Phi-3.5 and Phi-4-mini's longrope and partial_rotary_factor); granite (embedding_multiplier, residual_multiplier, attention_multiplier in place of 1/sqrt(head_dim), logits divided by logits_scaling); stablelm (LayerNorm with bias, use_qkv_bias, use_parallel_residual, per-head qk_layernorm); starcoder2 (LayerNorm with bias, use_bias on every projection, a plain c_fc/c_proj feed-forward, sliding window on every layer); cohere and cohere2 (Command R: LayerNorm without bias, a parallel block, interleaved rotation, logits times logit_scale, per-head use_qk_norm; Cohere 2 attends through sliding_window on all but every sliding_window_pattern-th layer and rotates only on the sliding ones); nemotron (LayerNorm with bias stored as an offset from one, a plain squared-ReLU feed-forward, mlp_bias); olmo2 and olmo3 (no norm before a sublayer, a norm over each sublayer's output, query and key norms over the whole projection; OLMo 3 adds sliding-window layers from layer_types); smollm3 (no rotary embedding on the layers no_rope_layers marks 0); gemma (1 + weight norms, embedding scaled by sqrt(hidden_size), tanh GELU gate, tied embeddings); gemma2 (Gemma's conventions plus norms after attention and feed-forward, query_pre_attn_scalar, attention and final logit soft-capping, sliding window on even layers); gemma3_text (Gemma 2's block with per-head query and key norms, sliding-window layers from layer_types or sliding_window_pattern, and rope_local_base_freq as their rotary base). Any family may state head_dim, attention_bias, layer_types, a hidden_act among silu, gelu, gelu_pytorch_tanh, gelu_new, gelu_fast and relu2, a partial_rotary_factor (only that share of each head rotates) and a rope_scaling of type llama3, linear or longrope (short factors while the sequence is within original_max_position_embeddings, long ones once it goes past, both tables scaled by the attention factor). The norm epsilon is read from rms_norm_eps, layer_norm_eps, norm_epsilon, norm_eps or layer_norm_epsilon, whichever the family uses. A plain feed-forward has no gate, so an adapter spec naming gate is refused on those families before any adapter is built. Coverage is measured against vLLM's list of text-only architectures; this list is the only place the families are enumerated, and every refusal names the same list.

Mixture-of-experts families replace the feed-forward with a router and experts: mixtral (Mistral's block, block_sparse_moe.gate and experts.{e}.w1/w3/w2, num_local_experts), qwen2_moe (Qwen2's block, mlp.gate, mlp.experts.{e}, moe_intermediate_size, and a shared_expert scaled by the sigmoid of shared_expert_gate on every token), qwen3_moe (Qwen3's block with mlp.gate and mlp.experts.{e}), olmoe (query and key norms over the whole projection, then every query, key and value component clamped to ±clip_qkv when it is stated), granitemoe (Granite's multipliers, block_sparse_moe.router.layer, and every expert stacked in input_linear and output_linear) and dbrx (everything below transformer: wte, blocks.{i}.norm_attn_norm.norm_1, attn.Wqkv clamped to ±clip_qkv, attn.out_proj, norm_2, ffn.router.layer, every expert's rows stacked in ffn.experts.mlp.w1, v1 and w2, and norm_f; LayerNorms without bias; the settings in attn_config and ffn_config are read beside the top-level ones, kv_n_heads and moe_top_k included; moe_normalize_expert_weights 1 renormalises the chosen weights, and any other power is refused). Each token goes to the num_experts_per_tok experts the router's softmax ranks highest; their weights are renormalised to sum to one when the family does (norm_topk_prob, always for Mixtral and GraniteMoE). Qwen MoE layers listed in mlp_only_layers, or whose position is not a multiple of decoder_sparse_step, keep a dense feed-forward. Adapters on these families sit on attention only: a gate, up or down target is refused at load and at merge.

Further dense families on the same decoder: olmo (OLMo 1: a LayerNorm with neither scale nor bias, epsilon one in a hundred thousand as Transformers fixes it; query, key and value clamped to ±clip_qkv when it is stated); exaone4 (OLMo 2's post-norm block with per-head query and key norms; when it has sliding-window layers, its global layers apply no rotary embedding); internlm3 (qkv_bias on query, key and value, bias on the output); seed_oss (attention_bias, attention_out_bias, mlp_bias); arcee (a plain up_proj/down_proj feed-forward, squared ReLU); ernie4_5 (use_bias on every projection, interleaved rotation, tied embeddings); minicpm (embedding times scale_emb, each sublayer times scale_depth / sqrt(layers), logits times dim_model_base / hidden_size); orion (LayerNorm with bias); glm and glm4 (a fused gate_up_proj, attention_bias on query, key and value, interleaved rotation over the partial_rotary_factor share of each head; GLM-4-0414 adds post_self_attn_layernorm and post_mlp_layernorm over the two sublayers' outputs).

jais2 (Jais 2) is Arcee's plain up_proj/down_proj feed-forward under LayerNorms with bias; attention_bias and mlp_bias default to true and hidden_act to squared ReLU, as Transformers' Jais2Config defines them. For every family, a config written by Transformers 5 that states its rotation as one rope_parameters object has its rope_theta read as the rotary base and, unless its rope_type is default, the object read as the rotary scaling; a top-level rope_theta or rope_scaling takes precedence.

Families with their own tensor paths: gpt_neox (GPT-NeoX and Pythia: everything below gpt_neox, embed_in and embed_out, final_layer_norm, a query_key_value whose rows go head by head — each head's query, key and value in turn — gathered into three projections at load and merged back head by head, biased projections, a plain dense_h_to_4h/dense_4h_to_h feed-forward, parallel blocks with a norm for each half unless use_parallel_residual is false, and rotation of the rotary_pct share of each head with rotary_emb_base); gptj (GPT-J: everything below transformer, wte, h.{i}, ln_1 before both halves of a parallel block, unbiased attn.q_proj/k_proj/v_proj/out_proj, a biased fc_in/fc_out feed-forward and head, ln_f, and interleaved rotation over rotary_dim). GPT-style key names — n_embd, n_layer, n_head, n_positions, n_inner (four times the residual width when null), rotary_emb_base — are read where Llama's are missing.

Families whose positions are not rotations: gpt2 (a learned wpe table added to the embedding, h.{i} with ln_1 and ln_2, one biased c_attn holding query, key and value, c_proj, a biased plain mlp.c_fc/mlp.c_proj feed-forward, ln_f; every projection stored inputs-first as Conv1D stores it, laid out once at load and merged back in that orientation; a scale_attn_by_inverse_layer_idx is refused); gpt_bigcode (StarCoder and SantaCoder: GPT-2's layout with ordinary linear weights and one key-value head when multi_query); opt (everything below model.decoder, a learned embed_positions read two rows in, self_attn_layer_norm and final_layer_norm in each pre-norm block, out_proj, a ReLU fc1/fc2; post-norm do_layer_norm_before: false and a word_embed_proj_dim unlike the model width are refused); bloom (ALiBi — each head adds its slope times the key's position to the scores — a word_embeddings_layernorm after the embedding, a head-interleaved self_attention.query_key_value, the tanh GELU; apply_residual_connection_post_layernorm is refused). A checkpoint saved from the base model without its root prefix (GPT-2's h.0… rather than transformer.h.0…) loads and merges the same way. GPT-2's n_embd, n_layer, n_head, n_positions, n_inner, BLOOM's n_embed, OPT's ffn_dim and activation_function are read where Llama's keys are missing.

falcon keeps BLOOM's tensor names without the embedding norm and groups its query_key_value rows by key-value head: one group for the multi-query models (multi_query), num_kv_heads groups for the new decoder architecture, one group per head otherwise. Its blocks are parallel when parallel_attn or new_decoder_architecture says so — with ln_attn and ln_mlp in the new architecture unless num_ln_in_parallel_attn is one — the feed-forward is the exact GELU, bias sets every projection's bias, and alibi replaces rotation with ALiBi added before the scores are divided by the head width, as Falcon does. mpt reads everything below transformer.blocks (norm_1, attn.Wqkv, attn.out_proj, norm_2, ffn.up_proj, ffn.down_proj, norm_f), drops every bias under no_bias, uses the exact GELU and an inner width of expansion_ratio times d_model, and takes ALiBi or a learned wpe from attn_config.alibi; an attn_config with qk_ln, clip_qkv, softmax_scale, an attn_type other than multi-head, or an alibi_bias_max other than eight is refused, as is ALiBi over a head count that is not a power of two.

Multi-head latent attention: deepseek_v2, deepseek_v3 (DeepSeek-V2, V2-Lite, V3 and Kimi-K2 checkpoints in unquantized form) and minicpm3 state kv_lora_rank, q_lora_rank, qk_nope_head_dim, qk_rope_head_dim and v_head_dim. The query passes through q_a_proj, q_a_layernorm and q_b_proj (or one q_proj without a query rank); the key and value through kv_a_proj_with_mqa, whose last qk_rope_head_dim columns are a rotated key part shared by every head, then kv_a_layernorm and kv_b_proj, which yields each head's position-free key part and its value, v_head_dim wide. Only the rotated part rotates — adjacent pairs for DeepSeek (rope_interleave, on by default), halves for MiniCPM3 — and DeepSeek sharpens the scores by the square of the YaRN temperature mscale_all_dim gives. DeepSeek's feed-forward is a mixture of n_routed_experts experts plus n_shared_experts shared ones added to every token, dense before first_k_dense_replace and off moe_layer_freq; experts are scored by scoring_func (softmax or sigmoid), chosen by topk_method (greedy; group_limited_greedy keeps the topk_group best of n_group groups ranked by their best expert; noaux_tc ranks groups by their best two and adds e_score_correction_bias only to choose), and weighted by routed_scaling_factor (always on V3, on V2 only without renormalisation). MiniCPM3 keeps MiniCPM's muP scales. Adapters on latent attention sit on the output projection, and on the query when it has no bottleneck. Any family may also state a rope_scaling of type yarn: frequencies that turn more than beta_fast times over original_max_position_embeddings keep their value, those turning fewer than beta_slow times are divided by factor, a linear ramp blends the band between, and the tables are scaled by the attention factor (mscale over mscale_all_dim when both are stated).

State-space models: mamba and falcon_mamba have no attention. Each block below backbone.layers is a norm and a selective state-space mixer: in_proj splits the input into a stream and a gate, a causal depthwise conv1d over the last conv_kernel stream values and SiLU follow, x_proj yields a step, an input matrix and an output matrix (Falcon-Mamba normalises each without a scale, mixer_rms_eps), dt_proj and softplus turn the step into a per-channel rate, the state_size-wide state decays by exp(rate · -exp(A_log)) and takes in the input, its read-out plus D times the stream is gated by SiLU of the gate, and out_proj returns to the model width. The scan runs token by token; between decode steps the convolution's last inputs and the state are carried in the layer's recurrent-state cache slot. Positions enter only through the recurrence, so nothing is rotated. time_step_rank: auto is the width over sixteen, rounded up, and expand times the width stands in for a missing intermediate_size. jamba interleaves the same mixer (mamba, sized by mamba_expand, mamba_d_state, mamba_d_conv and mamba_dt_rank, with weighted dt_layernorm, b_layernorm and c_layernorm) with attention on every attn_layer_period-th layer from attn_layer_offset — attention with no position signal at all — and gives every layer a pre_ff_layernorm and a feed_forward, routed over num_experts by softmax without renormalisation on every expert_layer_period-th layer from expert_layer_offset. These models are steered like any other; adapter training on them is refused, because their state-space layers have no attention or feed-forward projection.

glm4_moe (GLM-4.5 and 4.6) is GLM's attention — biased query, key and value, per-head norms under use_qk_norm, rotation by halves over partial_rotary_factor — with DeepSeek-V3's router, which its configs leave implicit: sigmoid scores, noaux_tc selection with e_score_correction_bias, n_shared_experts shared experts and first_k_dense_replace dense layers. internlm2 is Llama's block under InternLM2's names — model.tok_embeddings, attention.wqkv holding each key-value group's query heads, key and value in turn, attention.wo, feed_forward.w1/w3/w2, attention_norm, ffn_norm, and output as the head — with bias on the attention projections. exaone (EXAONE 3 and 3.5) is Llama's block below transformer.h: ln_1, attn.attention.q_proj/k_proj/v_proj/out_proj, ln_2, mlp.c_fc_0 (gate), mlp.c_fc_1 (up), mlp.c_proj (down), ln_f, and num_layers for the depth.

hunyuan_v1_dense is Llama's block with attention_bias and mlp_bias, and, under use_qk_norm, per-head query and key norms named query_layernorm and key_layernorm that apply after the rotation rather than before it. Its rope_scaling of type dynamic with an alpha is a fixed rotary base, rope_theta * alpha^(d / (d - 2)) over the head width d; a dynamic scaling without alpha is the plain rotation within max_position_embeddings.

Mamba-2: mamba2 (Mamba-Codestral) uses the structured multi-head mixer. in_proj yields the gate, the stream, the input and output matrices B and C (one pair per each of n_groups groups) and a step per head; stream, B and C go through a causal depthwise conv1d and SiLU; each of num_heads heads of head_dim scans a [head_dim, state_size] state with one scalar decay exp(Δ·-exp(A_log)), where Δ is softplus of the step plus dt_bias, clamped to time_step_limit; the read-out plus D times the stream is multiplied by SiLU of the gate, RMS normalised over each group's share with the stored norm scale, and projected by out_proj. bamba interleaves that mixer (mamba, sized by mamba_n_heads, mamba_d_head, mamba_n_groups, mamba_d_state and mamba_d_conv) with attention on the layers attn_layer_indices lists, rotated over attn_rotary_emb components of each head, and puts pre_ff_layernorm and a feed_forward after every layer. Configs Transformers wrote with Python's Infinity, -Infinity or NaN (Mamba-2's time_step_limit is [0.0, Infinity]) are read with those values as unstated.

Gemma 3's image-text checkpoints (model_type gemma3, such as gemma-3-4b-it) load as the gemma3_text language model their text_config describes, with eos_token_id, bos_token_id, tie_word_embeddings and quantization_config taken from the outer config when the nested one leaves them out. Every weight is read from below language_model, merges write back there, and the vision tower is never mapped. Gemma 4's (gemma4, such as gemma-4-E4B-it, and gemma4_unified, such as gemma-4-12B-it) load the same way as gemma4_text and gemma4_unified_text, from below model.language_model; their vision and audio towers are never mapped. A wrapper config without text_config is refused as an unsupported architecture. Configs that Transformers 5 writes with one rope_parameters object per layer type are read as the full_attention rotation, with the sliding_attention base as rope_local_base_freq; a scaled sliding_attention rotation is refused.

gemma4_text and gemma4_unified_text (Gemma 4) keep Gemma 3's norms around both sublayers and its per-head q_norm and k_norm, but every norm scales by its weight itself, scores are not divided by the head width, each head's value passes a scale-free RMS norm, and each layer's output is multiplied by its stored layer_scalar. Layers alternate sliding_window and full attention as layer_types lists. Full-attention heads are global_head_dim wide; under attention_k_eq_v they have num_global_key_value_heads key-value heads and no v_proj, the value being the key projection before its norm. Sliding layers rotate by the sliding_attention base over head_dim; full layers by the full_attention rotation over global_head_dim, which for rope_type proportional rotates only the first partial_rotary_factor share of the frequency pairs and passes the rest through, as Transformers' _compute_proportional_rope_parameters does. With num_kv_shared_layers, the last that many layers project only their query and reuse the keys and values of the last earlier layer of their kind. With hidden_size_per_layer_input (the E models), embed_tokens_per_layer times its width's square root, plus per_layer_model_projection of the scaled embeddings times the hidden width's inverse square root through per_layer_projection_norm, the sum times 2^-0.5, gives each layer an input that per_layer_input_gate, the GELU, per_layer_projection and post_per_layer_input_norm add to its output. With enable_moe_block, num_experts experts in experts.gate_up_proj and down_proj run beside the dense feed-forward: the router reads the residual through a scale-free norm times router.scale and the hidden width's inverse square root, keeps the top top_k_experts of its softmax renormalised and times router.per_expert_scale; the experts read it through pre_feedforward_layernorm_2; and the dense output through post_feedforward_layernorm_1 and the experts' through post_feedforward_layernorm_2 are added before post_feedforward_layernorm. Adapters train only on the dense feed-forward (gate, up, down); attention targets are refused because full-attention heads are wider than sliding ones. use_bidirectional_attention: "all", use_double_wide_mlp, and a config without global_head_dim, layer_types or a sliding-window rotation base are refused.

mimo (MiMo) is Qwen2 with next-token-prediction layers under model.mtp_layers, which a forward never reads. mellum (Mellum) is Qwen3-MoE with sliding_window layers as layer_types lists them, a dense feed-forward on the layers mlp_layer_types calls dense, and one rotation per layer kind: its full-attention layers rotate by the full_attention YaRN rotation and its sliding layers by the plain sliding_attention base, which any family stating rope_parameters per layer kind now gets. flex_olmo (FlexOlmo) is OLMo 2's post-norm block with whole-projection query and key norms and OLMoE's num_experts experts as its feed-forward. granitemoeshared adds shared_mlp, shared_intermediate_size wide, beside GraniteMoE's experts; granite_swa and granitemoe_swa add to Granite and GraniteMoE (with that shared expert when its width is above zero) a learned sink per head (self_attn.sinks), sliding-window layers as layer_types lists them (every layer but each fourth from the first without it), and layer_rope_theta, where zero leaves a layer unrotated; the stated bases must rotate full-attention layers by rope_theta and sliding layers by one base, or the config is refused. Every Granite family reads mlp_bias. hyperclovax (HyperCLOVA X) is Granite whose logits are multiplied by logits_scaling rather than divided, with post_norm1 and post_norm2 over each sublayer's output unless use_post_norm is false. glm4_moe_lite (GLM-4.7-Flash) is DeepSeek-V3's latent attention and sigmoid, noaux_tc router; a dense feed-forward runs on the layers mlp_layer_types calls dense, or the first first_k_dense_replace. iquestcoder (IQuest-Coder) is Llama with mlp_bias, clip_qkv, and Qwen2's sliding window from max_window_layers under use_sliding_window. vaultgemma (VaultGemma) is Gemma 2 without the norms over each sublayer's output.

apertus (Apertus) has per-head q_norm and k_norm, attention_layernorm and feedforward_layernorm before its two sublayers, and a plain mlp.up_proj, activation, mlp.down_proj feed-forward. Its activation is xIELU (hidden_act: xielu), whose parameters each layer keeps in mlp.act_fn: alpha_p · x² + beta · x above zero and (expm1(min(x, eps)) − x) · alpha_n + beta · x elsewhere, with alpha_p the softplus of the stored value and alpha_n beta plus the softplus of its stored value, as Transformers' XIELUActivation computes them. exaone_moe (K-EXAONE) is EXAONE 4's attention — per-head query and key norms, and no rotation on its full-attention layers when it has sliding-window ones — in a pre-norm block; the layers mlp_layer_types calls sparse route over num_experts experts with DeepSeek-V3's sigmoid, noaux_tc router, whose selection bias sits at mlp.e_score_correction_bias, and add num_shared_experts shared experts; the other layers run a dense feed-forward.

solar (Solar Pro) is Llama's block with block skip connections: before the layers bskcn_1 and bskcn_2 list, the hidden state is kept, and before the layers bskcn_3 and bskcn_4 list it becomes w · kept + (1 − w) · hidden with the first and second kept state, w being the inference weight, the second of bskcn_tv. step1 (Step1, Step-Audio's language model) is Llama's block with num_attention_groups key-value heads and no rotation; positions enter as −slope · sqrt(distance) added to the scaled scores, with ALiBi's slopes, measured from each query's absolute position. telechat3 (TeleChat3) is Llama's block with attention_bias and mlp_bias; its rope_type telechat3-yarn is YaRN whose temperature grows by 0.07 · ln(factor) where YaRN's grows by 0.1 · ln(factor). param2moe (Param2-MoE) is Ling 2.0's block, sigmoid router with mlp.gate.expert_bias and shared experts.

PanguEmbedded (openPangu-Embedded) is Llama's block with bias on every attention projection (qkv_bias, when stated, for query, key and value), mlp_bias, and under sandwich_norm norms over each sublayer's output (post_attention_layernorm, then pre_mlp_layernorm before and post_mlp_layernorm after the feed-forward). olmo_hybrid (OLMo Hybrid) runs Gated DeltaNet (linear_attn) on the layers layer_types calls linear_attention — each fourth from the fourth attending when it lists none — in a pre-norm block: separate q_proj, k_proj and v_proj through one causal convolution (or OLMo-core's q_conv1d, k_conv1d and v_conv1d, joined in that order), a write strength of 2 · sigmoid(b_proj(x)) under linear_allow_neg_eigval (sigmoid otherwise), a decay from a_proj, A_log and dt_bias, and the read-out normed by o_norm (epsilon 1e-5, as Transformers fixes it), gated by SiLU of g_proj and projected by o_proj. Its other layers are OLMo 3's post-norm attention blocks, rotated only when rope_parameters states a rope_theta. A family whose layer_types list recurrent mixers reads them as which layers run which mixer, never as sliding windows.

hy_v3 (HY V3, Hy3) has per-head q_norm and k_norm, a dense feed-forward on the layers mlp_layer_types calls dense (or the first first_k_dense_replace), and elsewhere num_experts experts under mlp.experts, chosen by sigmoid score of mlp.router.gate plus mlp.expert_bias, weighted by the sigmoid scores renormalised under route_norm and scaled by router_scaling_factor, beside num_shared_experts shared experts in mlp.shared_mlp.

Looped models run their stored layers more than once. nanbeige (Nanbeige) is Llama's block, with per-head q_layernorm and k_layernorm under qk_layernorm, whose layers run num_loops times, the final norm closing every pass unless skip_loop_final_norm; its n-gram embeddings, hyper-connections, split loops, shared loop caches and depth attention are refused. iquestloopcoder (IQuest-LoopCoder) is Llama's block whose layers run loop_num times; on every later pass a layer's attention is g · global + (1 − g) · local, global over the first pass's keys and values of the same layer and local over the pass's own within loop_window_size, with g = sigmoid(W_h · q_h + b_h) per head of the rotated query from model.gate_projections.{layer}. A looped model's layer count is every pass's layers — Nanbeige4.2-3B's 22 stored layers are 44 — so each pass has its own cache slots and its own steering and capture layers. Adapter training on a looped model is refused, since one adapter would correct every pass of its layer.

nemotron-nas (DeciLM: Llama-3.3-Nemotron Super and Ultra) is Llama's block in which every layer follows its own block_configs entry: num_attention_heads / n_heads_in_group key-value heads and a feed-forward 2 · ffn_mult · hidden_size / 3 wide, rounded up to a multiple of 256 (or the entry's intermediate_size). An entry whose attention or ffn is no_op has no such half — no norm, no weights — and the residual passes through it. replace_with_linear, sparsify, attention window_length and num_sink_tokens, and a per-layer hidden_act other than the model's are refused, and so is adapter training, since no projection has one shape on every layer.

deepseek_v32 (DeepSeek-V3.2) and glm_moe_dsa (GLM-5) are DeepSeek-V3's latent attention and sigmoid router with DeepSeek Sparse Attention: once a layer sees more than index_topk keys, each query attends only to the keys its indexer (self_attn.indexer) scores highest. The indexer reads the latent query through wq_b and the hidden state through wk and the LayerNorm k_norm, rotates the first qk_rope_head_dim components of each (by halves for DeepSeek, by adjacent pairs under GLM-5's indexer_rope_interleave), and scores a key as Σ_h w_h · relu(q_h · k / sqrt(index_head_dim)) with w = weights_proj(x) / sqrt(index_n_heads). With no more keys than index_topk, every key is kept and the attention is the dense one; the indexer's keys are still cached. GLM-5's shared layers (indexer_types, index_topk_pattern, or index_topk_freq and index_skip_topk_offset) have no indexer and reuse the last indexed layer's choice. DeepSeek's own V3.2 checkpoints declare FP8 quantization_config and are refused; their BF16 exports load.

deepseek_v4 (DeepSeek-V4), in Transformers' tensor layout or DeepSeek's own (embed, layers.{i}.attn.wq_a, ffn.experts, hc_attn_fn, with or without a model. root), keeps hc_mult residual streams. Before each sublayer a hyper-connection (attn_hc, ffn_hc: fn, base, scale) reads the streams flattened and RMS-normed without a scale into three weights: pre collapses the streams into the sublayer's input, post spreads its output back over them, and comb, a softmax made doubly stochastic by hc_sinkhorn_iters Sinkhorn-Knopp rounds, mixes the incoming streams into the outgoing ones; hc_head collapses the last streams before the final norm. Ster reads and steers the streams' mean: --layer captures their mean and a steering vector is added to every stream. Attention takes its queries from a latent (q_a_proj, q_a_norm, q_b_proj) RMS-normed per head without a scale, and one key-value head (kv_proj, kv_norm) that is both key and value; the trailing qk_rope_head_dim channels of each head rotate pair by pair, at rope_theta on the sliding layers and at compress_rope_theta with YaRN (magnitude one) on the compressed ones. A learned sink logit per head (sinks) joins every softmax, every key within sliding_window is seen, and the output's rotated slice is rotated back at the query's position before o_a_proj projects each of o_groups head groups to o_lora_rank and o_b_proj mixes them. On the heavily_compressed_attention layers a compressor (self_attn.compressor) turns every window of the heavy rate (compress_rates, 128 by default) into one entry — kv_proj(x) weighted by the window's softmax of gate_proj(x) + position_bias, summed, normed by kv_norm and rotated at the window's first position — and a query also sees every entry whose window closed at or before it. On the compressed_sparse_attention layers the compressor runs over windows of the sparse rate (4 by default) with two series per token, each entry mixing its own window with the one before, and the Lightning Indexer (compressor.indexer: its own such compressor at index_head_dim, q_b_proj from the latent query, scorer.weights_proj) keeps for each query its index_topk best closed entries by Σ_h w_h · relu(q_h · k). Tokens past the last whole window, the entries made so far and the last window's first series are kept per layer across decode calls, so a decode continued from the cache sees what the same text read whole sees. The feed-forward routes every layer: experts (mlp.experts.{e}.w1, w3, w2, or stacked gate_up_proj and down_proj) scored sqrtsoftplus, chosen with mlp.gate.e_score_correction_bias added — on the hash_moe layers (mlp_layer_types, else the first num_hash_layers) by the token's row of mlp.gate.tid2eid — renormalised, scaled by routed_scaling_factor, beside one shared expert; every SwiGLU is silu(min(gate, swiglu_limit)) · clamp(up, ±swiglu_limit). Older configs' compress_ratios (0, 4, 128) are read as layer kinds, and a head stored as head is read as the head. Adapters are refused; the published DeepSeek-V4 weights are FP8 and FP4 and are refused before download. `ster parity` against Transformers' own forward pass on malaiwah/deepseek-v4-tiny-random-bf16 (BF16, DeepSeek's own layout) measures 2.8 % relative error with every mean cosine above 0.999, and a decode continued from the cache matches the same text read whole.

glm5_next_text (GLM-5-Next, also inside glm5_next below model.language_model) keeps hc_mult residual streams mixed by the same hyper-connections as DeepSeek-V4 (attn_hc and ffn_hc, or the checkpoint's own hc_attn_fn, hc_attn_base, hc_attn_scale and hc_ffn_*), and averages the last streams before the final norm instead of collapsing them through hc_head; --layer captures the streams' mean and a steering vector is added to every stream. The layers layer_types marks linear_attention run Kimi Delta Attention from linear_attn_config (num_heads, head_dim, short_conv_kernel_size; under safe_gate the log-decay is gate_lower_bound · sigmoid(exp(A_log) · (f + dt_bias))). The deepseek_sparse_attention layers run latent attention (q_a_proj, q_a_layernorm, q_b_proj, kv_a_proj_with_mqa, kv_a_layernorm, kv_b_proj) with qk_rope_head_dim 0, so no component rotates and positions come from the causal order alone, behind a pooled indexer: every index_kpool consecutive keys form one pool whose key is the softmax-weighted sum of theirs by index_kpool_compress_gate(x) + index_kpool_compress_ape, each query keeps its index_topk / index_kpool best pools that closed at or before it, and under index_kpool_always_select_tail also the keys of its own unfinished pool. The indexer's keys and gates are kept per layer across decode calls. The feed-forwards mlp_layer_types marks sparse route DeepSeek-V3's way (sigmoid scores, mlp.gate.e_score_correction_bias, n_group and topk_group, routed_scaling_factor) beside n_shared_experts shared experts; the others are dense, and every SwiGLU is silu(min(gate, swiglu_limit)) · clamp(up, ±swiglu_limit). Adapters are refused. `ster parity` against Transformers' own forward pass on malaiwah/glm5-next-tiny-random-bf16 measures 2.0 % relative error with a mean cosine of 0.9998.

hy_v4 (Tencent's HY V4, Hy4) is read as vLLM serves it. Attention is DeepSeek-V3's latent attention rotating adjacent pairs, as vLLM rotates its Megatron-layout checkpoint, with a learned sink logit per head in every softmax under learnable_sink and, under gated_mla with gating_type elementwise, its output multiplied by the sigmoid of self_attn.linear_gate(x) before o_proj; another gating_type is refused. DeepSeek Sparse Attention keeps each query's index_topk keys; the layers indexer_types marks shared reuse the last indexed layer's choice, and the indexer rotates the last qk_rope_head_dim components of each head rather than the first. Under enable_ihc the residual is hc_mult streams joined by independent hyper-connections (hc_attn_layer and hc_mlp_layer): each reads the streams flattened and RMS-normed into pre, sigmoid(·) + hc_eps, which collapses them into the sublayer's input, and post, hc_magnitude · sigmoid(·) + hc_eps, which adds the sublayer's output to every stream; no matrix mixes the streams, and hc_head collapses the last ones before the final norm. --layer captures the streams' mean and a steering vector is added to every stream. The layers mlp_layer_types marks dense keep a dense feed-forward; elsewhere n_routed_experts experts stacked in mlp.experts.gate_up_proj and mlp.experts.down_proj are scored by a sigmoid of mlp.gate, chosen with mlp.gate.e_score_correction_bias added, renormalised under norm_topk_prob and scaled by routed_scaling_factor, beside n_shared_experts shared experts. The routed experts are SwiGLU clamped by swiglu_limit (unclamped at zero) whatever hidden_act says; the dense and shared feed-forwards run hidden_act unclamped. soft_logits_capping soft-caps the logits at soft_logits_capping_logits. A key HYV4Config gives a default (enable_ihc, gated_mla, learnable_sink, hc_mult 4, hc_magnitude 2, hc_eps, index_n_heads 16, n_shared_experts 1, routed_scaling_factor, swiglu_limit 10, one leading dense layer, an indexer on layer 0 and every fourth layer from layer 1) reads as that default when the config leaves it out. The release's tensor names (hc_attn_layer.hc_pre.hc_fn, hc_head.hc_head_fn, learnable_sink_param, linear_gate) and those Transformers' own writer produces (hc_pre.hc_attn_layer.hc_fn, mlp.linear_gate) are both read through Transformers' hy_v4 renames. The mtp_layers are never mapped. The head is computed at the weights' precision and its logits cast to F32, where vLLM accumulates it in F32 (enable_lm_head_fp32); at --precision f32 the two agree.

dots3_note (Dots3-Note) is read as vLLM serves it: DeepSeek-V3.2's block, with two latent attentions. The layers layer_types calls full_attention keep num_attention_heads heads of DeepSeek's latent attention (q_lora_rank, kv_lora_rank, qk_nope_head_dim, qk_rope_head_dim, v_head_dim) and DeepSeek Sparse Attention's indexer, which rotates adjacent pairs unless indexer_rope_interleave is false. The sliding_attention layers run a dense latent attention of their own, swa_num_attention_heads heads built from swa_q_lora_rank, swa_kv_lora_rank, swa_qk_nope_head_dim, swa_qk_rope_head_dim and swa_v_head_dim, scored over their own head width, rotated by swa_rope_theta and attending through sliding_window_size. Both kinds pass the rotated key part they share across heads through k_rope_only_layernorm before it rotates, multiply their bottlenecks' normed outputs by sqrt(hidden_size / rank) of their own ranks under apply_mla_qkv_lora_rescale, and multiply each head's output by the sigmoid of its g_proj logit before o_proj. Both rotate by their plain base, as vLLM does whatever rope_scaling states. The experts are DeepSeek-V3's, chosen by noaux_tc with e_score_correction_bias over one group when the config states no n_group and topk_group, as Dots3NoteConfig sets them; the first first_k_dense_replace layers stay dense. The image and audio towers and the MTP layer are never mapped. An attention_gate_type or swa_attention_gate_type other than headwise, or sliding-window heads rotating another width than the full-attention ones, is refused by name, and adapter attention targets are refused because the two kinds of layer differ in shape.

minimax_m3_vl_text (MiniMax-M3's text decoder) is read from the text_config of a minimax_m3_vl checkpoint, its weights below language_model and its vision tower never mapped; a config that leaves out model_type and names MiniMaxM3SparseForCausalLM is the same family. It is read as vLLM's MiniMaxM3SparseForCausalLM and Transformers' MiniMaxM3VLTextModel run it. Every norm scales by its weight plus one (use_gemma_norm), queries and keys are normed per head (qk_norm_type per_head) before rotating over partial_rotary_factor of each head, and every feed-forward is SwiGLU-OAI (hidden_act swigluoai): (clamp(up, ±swiglu_limit) + 1) · g · sigmoid(swiglu_alpha · g) with g = min(gate, swiglu_limit). The layers moe_layer_freq marks zero keep a dense mlp of dense_intermediate_size; the rest score num_local_experts experts under block_sparse_moe (gate, experts.{e}.w1/w3/w2, intermediate_size wide) by scoring_func (sigmoid), choose num_experts_per_tok with block_sparse_moe.e_score_correction_bias added, renormalise, scale by routed_scaling_factor, and add n_shared_experts shared experts of shared_intermediate_size (block_sparse_moe.shared_experts). The layers sparse_attention_config.sparse_attention_freq marks run MiniMax Sparse Attention: self_attn.index_q_proj gives one index query per key-value head (sparse_num_index_heads, sparse_index_dim wide), index_k_proj one index key all of them share, both normed (index_q_norm, index_k_norm) and rotated like the main heads; a key scores as their plain product, a block of sparse_block_size keys as its best key the query may see, and each query of a key-value group attends only its group's sparse_topk_blocks best blocks, its own block and the sparse_local_block blocks ending there always among them. Blocks are counted from the first key slot, as Transformers counts them. While a sequence holds no more blocks than a query keeps, every key is kept and the layer is dense. Another hidden_act, attention_output_gate, use_gemma_norm or use_qk_norm false, another qk_norm_type or scoring_func, a sparse_score_type other than max, leading sparse_init_block blocks, an enabled index value branch (sparse_disable_index_value 0 on a sparse layer), or index heads other than one per key-value head, is refused by name. MiniMaxAI/MiniMax-M3 is BF16; its MXFP8 and NVFP4 releases are quantized and refused.

axk1 (A.X-K1) is DeepSeek-V3's latent attention and sigmoid router with post_mlp_layernorm over each routed layer's feed-forward output; its topk_method none is grouped top-k without a selection bias, each group ranked by its best expert. Experts saved by Transformers 5 — stacked in mlp.experts.gate_up_proj ([experts, 2 · width, hidden], gate rows first) and mlp.experts.down_proj — load in every family that otherwise keeps one module per expert under mlp.experts.{e}.

bailing_hybrid (Ling 2.5 and 3.0) is Ling 2.0's router and experts with two attentions under attention: DeepSeek's latent attention, rotating adjacent pairs under rope_interleave, on every layer_group_size-th layer, and linear attention on the others. Ling 2.5's linear attention is lightning attention: it takes each head's query, key and value from query_key_value (through SiLU under linear_silu), normalises query and key per head (query_layernorm, key_layernorm) under use_qk_norm, rotates the first partial_rotary_factor share of each head_dim-wide head by halves at rope_theta, scales the query by head_dim^-0.5, and keeps per head a state decaying by exp(-slope) with MiniMax's slopes scaled by 1 − layer / (layers − 1) + 1e-5. Its read-out is RMS normalised over group_norm_size equal groups, scaled by g_norm, multiplied by the sigmoid of g_proj and projected by dense. A config whose lightning heads rotate a different width than the latent heads, or whose num_kv_heads_for_linear_attn differs from its heads, is refused. Ling 3.0 (BailingMoeV3ForCausalLM, or any config stating short_conv_kernel_size) runs Kimi Delta Attention instead, on every layer that neither closes a layer_group_size group nor follows the last whole group: num_attention_heads heads of head_dim from q_proj, k_proj and v_proj through their own q_conv1d, k_conv1d and v_conv1d of short_conv_kernel_size taps, a decay per key channel from the full-rank f_proj — kda_lower_bound · sigmoid(exp(A_log) · (f + dt_bias)) under kda_safe_gate (on by default, the bound −5 when left out), Kimi's -exp(A_log) · softplus(f + dt_bias) otherwise — write strength sigmoid(b_proj(x)), and o_norm gated by the sigmoid of g_proj before o_proj. Its latent-attention layers multiply each head's output by the sigmoid of attention.g_proj under gated_attention_proj_granularity_type head_wise, and its experts take Step 3.5's per-layer clamps from expert_swiglu_limit_list and share_expert_swiglu_limit_list. Factored Kimi projections (no_kda_lora false or use_kda_lora), use_nGPT, value_norm, up_proj_norm, use_mla_nope and any other gate granularity are refused.

mimo_v2_flash and mimo_v2 (MiMo-V2-Flash, MiMo-V2.5) attend through a sliding_window on the layers hybrid_layer_pattern marks 1, with swa_num_key_value_heads key-value heads, a learned sink per head (self_attn.attention_sink_bias) under add_swa_attention_sink_bias and their own base swa_rope_theta; the other layers attend fully with num_key_value_heads heads and sinks only under add_full_attention_sink_bias. Query and key heads are head_dim wide with their first partial_rotary_factor share rotated by halves; value heads are v_head_dim wide and every value is multiplied by attention_value_scale. Query, key and value are separate projections, or stacked in qkv_proj under attention_projection_layout fused_qkv. Layers moe_layer_freq marks 1 run DeepSeek-V3's sigmoid router (mlp.gate.e_score_correction_bias) over n_routed_experts; the rest are dense. A config whose sliding-window heads differ in count or width from its full-attention heads is refused, and no attention projection takes an adapter because the two kinds differ in key-value heads.

step3p5 (Step 3.5) normalises with RMS norms scaled by 1 + weight throughout, query and key per head (q_norm, k_norm), and multiplies each head's output by the sigmoid of its self_attn.g_proj logit under use_head_wise_attn_gate. Its layers alternate full and sliding_window attention as layer_types lists; the sliding-window layers take their head and key-value head counts from attention_other_setting, the full-attention layers from num_attention_heads and num_attention_groups. Each kind rotates its own partial_rotary_factors share by halves at its own base from the per-layer rope_theta, rope_scaling reaches the full-attention layers only (yarn_only_types), and use_rope_layers can leave layers unrotated. The layers moe_layers_enum lists route over moe_num_experts experts stacked under moe by moe_router_activation scores that moe.router_bias moves to choose, renormalised under norm_expert_weight and scaled by moe_router_scaling_factor, beside share_expert; on a layer whose swiglu_limits or swiglu_limits_shared entry is not zero, the routed or shared expert computes min(silu(gate), limit) · clamp(up, ±limit). Per-layer lists longer than num_hidden_layers cover the multi-token-prediction layers, which Ster does not run. A kind whose layers list different rotated shares or bases, rotation scaling on sliding-window layers, or a clamped dense layer is refused.

k2_horizon (K2-Horizon) is Llama's block whose hidden-width RMS norms normalise each of layernorm_num_groups equal groups apart before the stored scale, with biased projections under attention_bias and, under attention_gate_func, attention's output multiplied by silu or softplus with β = ln 2 (log2(1 + 2^g)) of self_attn.gate_proj. The layers neither in mlp_only_layers nor off decoder_sparse_step route over num_experts experts (mlp.experts.{e}) by router_score_func scores that mlp.gate.bias moves to choose under moe_gate_bias, renormalised under norm_topk_prob and scaled by router_scaling_factor, beside num_shared_experts shared ones (mlp.shared_experts). Under mova_num_experts those layers have no v_proj: each value is the router-weighted sum of the SiLU outputs of mova_num_experts_per_tok value experts (self_attn.v_experts.{e}) chosen by self_attn.v_router the same way, and no value adapter is trained. A rotation narrower than the head (rope_head_dim) and query_key_norm are refused.

pangu_ultra_moe (openPangu-Ultra-MoE) is DeepSeek-V3's latent attention, rotating adjacent pairs, under its own key names: attention_q_lora_dim, attention_kv_lora_dim, attention_qk_dim, attention_qk_rope_dim and attention_v_dim are read as q_lora_rank, kv_lora_rank, qk_nope_head_dim, qk_rope_head_dim and v_head_dim, and num_routed_experts, num_shared_experts and num_dense_layers as n_routed_experts, n_shared_experts and first_k_dense_replace. Its router scores by sigmoid without a selection bias, renormalises the chosen weights (its config class defaults norm_topk_prob to true) and scales them by routed_scaling_factor; under sandwich_norm each sublayer's output passes post_attention_layernorm or post_mlp_layernorm and the feed-forward reads pre_mlp_layernorm.

longcat_flash (LongCat-Flash) stores num_layers layers that Ster runs as twice as many: each stored layer's halves 0 and 1 are input_layernorm.{h}, DeepSeek's latent attention self_attn.{h} (rotating adjacent pairs), post_attention_layernorm.{h} and a dense mlps.{h}, so --layer and the key-value cache count halves. Beside them, shortcut-connected experts read the first half's feed-forward input and join the residual at the end of the second half: mlp.router.classifier scores n_routed_experts experts expert_ffn_hidden_size wide and zero_expert_num identity experts by softmax, mlp.router.e_score_correction_bias moves the choice of moe_topk, and the chosen weights are scaled by routed_scaling_factor; an identity expert returns its input. Under mla_scale_q_lora and mla_scale_kv_lora the latent bottlenecks' normed outputs are multiplied by sqrt(hidden_size / q_lora_rank) and sqrt(hidden_size / kv_lora_rank). No adapter is trained on it.

longcat_flash_ngram (LongCat-Flash-Lite) is LongCat-Flash with hashed n-gram embeddings beside the token embedding. Its config may leave model_type out and name only LongcatFlashNgramForCausalLM in architectures, which Ster reads as longcat_flash_ngram. For each n-gram order from 2 to emb_neighbor_num and each of emb_split_num tables, the token and the order − 1 tokens before it hash to (token + Σ previous_d · vocab^d mod rows) mod rows, a table ngram_vocab_size_ratio · vocab_size + 2 · index + 1 rows tall (ngram_embeddings.embedders.{index}); each row passes ngram_embeddings.post_projs.{index}, and the token embedding and every projection are averaged. A token before the start of its run counts as zero, a run restarting after each eos_token_id; across decode steps the cache keeps each sequence's last emb_neighbor_num − 1 tokens.

sarvam_moe (Sarvam-30B) is Ling 2.0's block and router under Sarvam's name. sarvam_mla (Sarvam-105B) is DeepSeek's latent attention, rotating adjacent pairs, with a deepseek_yarn rotation (YaRN with DeepSeek's mscale ratio, the same scaling yarn gives) and a router over num_experts experts on the layers from first_k_dense_replace (one by default) at every moe_layer_freq-th: score_function scores (sigmoid by default) that mlp.gate.e_score_correction_bias moves under moe_router_enable_expert_bias (on by default), within the best topk_group of n_group groups when both are stated, renormalised unless norm_topk_prob is false and scaled by routed_scaling_factor (2.5 when left out), beside num_shared_experts shared experts moe_shared_expert_intermediate_size (else moe_intermediate_size) wide.

cohere2_moe (Cohere2-MoE, North-Mini-Code) is Cohere's parallel block — one input_layernorm before attention and the feed-forward side by side — with RMS norms, or LayerNorms without a bias when the config states no rms_norm_eps, and logits multiplied by logit_scale. Adjacent pairs rotate on the sliding-window layers layer_types lists, whose window is sliding_window + 1 keys as vLLM counts it, and under prefix_dense_sliding_window_pattern 1 on the dense prefix too; the other layers apply no rotation. The dense layers (mlp_layer_types, else the first first_k_dense_replace) are prefix_dense_intermediate_size wide; the rest route over num_experts experts intermediate_size wide by expert_selection_fn scores (sigmoid or softmax), renormalised under norm_topk_prob, beside num_shared_experts shared experts whose sum with the routed ones is halved under shared_expert_combination_strategy average. use_qk_norm and use_gated_activation false are refused.

deepseek (DeepSeek-MoE, the first DeepSeek mixture) is Llama's attention, rotating by halves, with DeepSeek's experts: n_routed_experts scored by softmax beside n_shared_experts shared ones, dense before first_k_dense_replace and off moe_layer_freq. ministral3 (Ministral 3, also as the text decoder inside mistral3, whose weights sit below language_model) is Mistral's block with Llama 4's query temperature: past every original_max_position_embeddings positions each query is multiplied by 1 + llama_4_scaling_beta · ln(1 + floor(position / original_max_position_embeddings)). FP8 Ministral 3 checkpoints declare quantization_config and are refused; the BF16 base checkpoints load.

Rnj-1 and Rnj-1.5 are gemma3_text checkpoints. Rnj-1.5 lists chunked_attention layers in layer_types: they attend by chunks of sliding_window positions, each query seeing the keys of its own chunk and of the one before it (vLLM's chunk_lookback of one), rotated by the full_attention rotation; a chunked_attention rotation that differs from it is refused. Masks for chunked layers follow the same rule in batched passes.

llama4_text (Llama 4, also as the text decoder inside llama4, whose weights sit below language_model) is Llama's attention with the layers no_rope_layers marks 0 (every fourth by default) unrotated and attending fully, and the rotating layers attending by chunks of attention_chunk_size positions, each query seeing only its own chunk. Under use_qk_norm the rotating layers' queries and keys pass a weightless RMS norm after the rotation; under attn_temperature_tuning the unrotated layers' queries at position p are multiplied by 1 + attn_scale · ln(1 + floor((p + 1) / floor_scale)). The layers moe_layers lists (else every interleave_moe_layer_step-th) route each token by sigmoid scores to num_experts_per_tok of num_local_experts experts stacked in feed_forward.experts.gate_up_proj and down_proj, each reading its input already multiplied by its score, beside feed_forward.shared_expert; the other layers are dense, intermediate_size_mlp wide.

qwen3_5_text and qwen3_5_moe_text (Qwen3.5 and Qwen3.5-MoE, also as the text decoders inside qwen3_5 and qwen3_5_moe, whose weights sit below model.language_model) are Qwen3-Next with its linear attention's projections stored apart: linear_attn.in_proj_qkv holds every query, then every key, then every value under one conv1d, and in_proj_z, in_proj_b and in_proj_a give the output gate, b and the decay input. The output gate on full attention follows attn_output_gate (on by default), the experts renormalise unless norm_topk_prob is false, and the multimodal rotation sections (mrope_section) rotate text as the plain rotation does.

afmoe (Arcee Trinity) puts norms around both sublayers (post_attention_layernorm over attention's output, pre_mlp_layernorm before the feed-forward, post_mlp_layernorm over it), normalises query and key per head, multiplies attention's output by the sigmoid of self_attn.gate_proj, and rotates only the sliding-window layers layer_types lists; under mup_enabled the embeddings are multiplied by sqrt(hidden_size). The layers from num_dense_layers route over num_experts experts (mlp.router.gate) by score_func scores that mlp.expert_bias moves to choose, renormalised under route_norm when they are sigmoids and scaled by route_scale, beside num_shared_experts shared experts.

Nemotron-H's mixture-of-experts layers (E) may route in a latent width: under moe_latent_size the routed experts work on mixer.fc1_latent_proj(x) and their weighted sum returns through mixer.fc2_latent_proj, while the router and the n_shared_experts shared experts read x. nemotron_h_puzzle (Nemotron Puzzle) is Nemotron-H below model rather than backbone, its layers listed in layers_block_type (mamba, attention, mlp, moe) and each moe layer's num_experts_per_tok and moe_intermediate_size stated in block_configs.

chatglm (GLM-4-9B, ChatGLM2 and ChatGLM3) keeps everything below transformer: embedding.word_embeddings, encoder.layers.{i} with self_attention.query_key_value (query, key and value stacked, biased under add_qkv_bias or add_bias_linear) and self_attention.dense, mlp.dense_h_to_4h (gate rows above up rows) and dense_4h_to_h, encoder.final_layernorm, and output_layer as the head. Its heads are kv_channels wide with multi_query_group_num key-value groups under multi_query_attention; the first half of each head rotates at base 10000 · rope_ratio, adjacent pairs under original_rope. Norms are RMS norms under rmsnorm and LayerNorms otherwise. A residual after the norm (apply_residual_connection_post_layernorm) or a missing final norm (post_layer_norm false) is refused.

Official chatglm checkpoints (GLM-4-9B-chat) publish no tokenizer.json, only GLM-4's tiktoken tokenizer.model: one base64(bytes) rank line per token id. Ster reads it as the byte-level BPE tiktoken runs: text is split by ChatGLM4Tokenizer.pat_str, a piece that is itself a token stays whole, and the rest merge pair by pair in rank order, the merges recovered from the ranks as Transformers' TikTokenConverter recovers them; added_tokens_decoder's special tokens follow the ranks, and [gMASK]<sop> starts every sequence encoded with special tokens. zai-org/glm-4-9b-chat answers Paris to What is the capital of France? Answer in one word. through its chat template, and continues The three primary colors are with red, blue, and green.

laguna (Laguna) normalises query and key per head and multiplies each head's output by the softplus of its self_attn.g_proj logit under gating per-head. Its layers alternate full and sliding_window attention as layer_types lists; each kind takes its head count from num_attention_heads_per_layer and its rotation from the per-layer-type rope_parameters — the full layers' YaRN over their partial_rotary_factor share, the sliding ones' plain rotation over theirs — and under swa_attention_sink_enabled the sliding layers carry a learned sink per head (self_attn.sink). The layers mlp_layer_types calls sparse route over num_experts experts by sigmoid scores that mlp.experts.e_score_correction_bias moves to choose, renormalised and scaled by moe_routed_scaling_factor, applied to the experts' input under moe_apply_router_weight_on_input, beside mlp.shared_expert. A kind with more than one head count and a router logit cap (moe_router_logit_softcapping) are refused.

muse_glimmer_text (Muse Glimmer, also as the text decoder inside muse_glimmer, whose weights sit below model.language_model while lm_head stays at the root) is Gemma 2's sandwich block — input_layernorm, post_attention_layernorm, pre_feedforward_layernorm, post_feedforward_layernorm, each storing its scale as an offset from one, the two output norms at post_norm_eps — with a weightless RMS norm after the embedding and a final norm that stores its scale as is. Unless use_qk_norm is false, every query and key passes a weightless per-head RMS norm and the query is multiplied by scale_query_by, else qk_scale_factor (divided by sqrt(head_dim) when it is at least that, the native schema). Unless use_attn_output_gate is false, attention's output is multiplied by the sigmoid of self_attn.gate_proj. The layers layer_types calls sliding_attention attend through sliding_window; layer_rope_theta gives each layer's base, zero leaving the full-attention layers unrotated. The logits are multiplied by output_multiplier and capped at final_logit_softcapping. A head stored beside a wrapped decoder rather than inside it is read from the checkpoint's root, for every wrapped family.

hyperclovax is also read as the text decoder inside hyperclovax_vision_v2 (HyperCLOVA X SEED Think 32B, vLLM's HCXVisionV2ForCausalLM): its text_config is the decoder's config and its weights sit below model.language_model (model.language_model.model.layers.{i}, model.language_model.lm_head); the vision and audio towers beside it are never mapped.

gemma3n_text (Gemma 3n, also as the text decoder inside gemma3n, below model.language_model) is Gemma 4's attention — per-head q_norm and k_norm, a scale-free norm over each value, scores left undivided, sliding_window layers as layer_types lists them rotating at rope_local_base_freq, the last num_kv_shared_layers layers reusing an earlier layer's keys and values — with its per-layer inputs and three additions. AltUp carries altup_num_inputs copies of the hidden state: after the scaled embedding, altup_projections.{i} make the extra copies, each rescaled to the first's root-mean-square; every layer predicts each copy from all of them (altup.prediction_coefs, routed by tanh(modality_router(router_norm(x) / hidden_size))), runs its block on the first prediction, corrects every copy by the block's change (correction_coefs + 1), and adds the per-layer input — gated by the first corrected copy times correct_output_scale — to every copy but the first; before the final norm altup_unembed_projections.{i} bring the copies back, rescaled, and all are averaged. Laurel adds linear_right(linear_left(x)) through post_laurel_norm to the normed input and averages it with attention's residual (divided by √2). On the layers whose activation_sparsity_pattern entry p is above zero, the feed-forward gate keeps only what lies above its mean plus Φ⁻¹(p) standard deviations. Steering and capture use the first copy. unsloth/gemma-3n-E2B-it answers Paris. to What is the capital of France? Answer in one word. through its chat template.

hrm_text (HRM-Text) stores two stacks of Llama layers, model.L_module.layers and model.H_module.layers, each num_layers_per_stack (else num_hidden_layers) long. Every layer's attn.gqkv_proj holds the attention gate's rows, then query, key and value; the gate's sigmoid multiplies attention's output before attn.o_proj, the feed-forward is a fused mlp.gate_up_proj, and every norm is an RMS norm with no scale. The stacks run H_cycles times as L_cycles passes of the low stack and one of the high stack: the high state starts as the embedding times embedding_scale and the low state as model.z_L_init; a pass reads the two states summed, ends in the stack's norm and replaces its own state; the last high state is the output. Every pass is its own run of Ster layers with its own key-value slots — 2 × (3 + 1) × 16 = 128 for HRM-Text-1B — so --layer counts passes, and adapters are refused because one stored layer runs on several passes. Under prefix_lm (on by default) a prompt read whole from position zero attends both ways within itself, as the model card asks for with token_type_ids of ones; decode steps and passes that score every position are causal. sapientinc/HRM-Text-1B answers Paris to the model card's direct few-shot format (<|im_start|><|object_ref_start|> … <|im_end|>).

plamo3 (PLaMo 3) is Llama's block under its own names — model.layers.layers.{i}, mixer.qkv_proj (query, key and value stacked), mixer.o_proj, mlp.gate_up_proj (gate rows above up rows) — with sandwich norms: pre_mixer_norm, pre_mlp_norm and the final norm store their scales as offsets from one, post_mixer_norm and post_mlp_norm as offsets from 1/5 and 5^-1.5, as modeling_plamo.py fixes them. Query and key pass per-head q_norm and k_norm before the rotation. Every sliding_window_pattern-th layer attends fully and rotates at rope_theta, by YaRN over initial_context_length positions (band 32 to 1, not rounded) when rope_scaling_factor is not one; the others attend through window_size and rotate at rope_local_theta. Under scale_embedding the embedding is multiplied by sqrt(hidden_size); a linear_type other than normal is refused. Every YaRN rotation now honours Transformers' truncate: false (PLaMo 3, GPT-OSS) leaves the band's bounds unrounded.

A checkpoint that publishes PLaMo's tokenizer.jsonl and no tokenizer.json has its tokenizer built from it: one [piece, score, kind] row per id, read as a unigram model whose split maximises the summed scores, with a character no piece covers spelled as its <0xXX> byte tokens, every CONTROL and UNKNOWN piece a special token, the stretches break_around_repeated_chars_threshold and break_around_consecutive_spaces_threshold in tokenizer_config.json cut out before the split, and the BOS token prepended under add_bos_token. A byte token is given the lowest score any piece has, so a spelled-out <0xXX> in text is split as text. Ties between equally scored splits may resolve differently from PLaMo's own Python tokenizer.

gpt_oss has biased query, key, value and output projections and a learned sink per head (self_attn.sinks): one extra logit in every query's softmax whose share is dropped rather than spent on a value. Its layers alternate full and sliding_window attention as layer_types lists, and its rotation is YaRN. Each feed-forward is a mixture of num_local_experts experts behind a biased mlp.router; the top num_experts_per_tok are renormalised, and each expert — stacked inputs-first in mlp.experts.gate_up_proj (gate and up columns interleaved) and down_proj, with biases — computes (clamp(up, ±limit) + 1) · g · sigmoid(1.702 g) with g = min(gate, limit) and limit its swiglu_limit. OpenAI's own GPT-OSS checkpoints declare MXFP4 quantization_config and are refused; their BF16 exports load.

lfm2 (LFM2) convolves on every layer but those full_attn_idxs (or layer_types' full_attention) names. A convolution layer is operator_norm, then conv.in_proj to B, C and x, a causal depthwise conv over B·x spanning conv_L_cache positions — continued between decode steps from the last inputs kept in the layer's cache slot — C times its output and conv.out_proj; then ffn_norm and feed_forward.w1/w3/w2. An attention layer has per-head q_layernorm and k_layernorm and out_proj. The feed-forward width is block_ff_dim, shrunk to two thirds, scaled by block_ffn_dim_multiplier and rounded up to block_multiple_of under block_auto_adjust_ff_dim; embedding_norm closes the stack. Adapter training on LFM2 is refused.

ernie4_5_moe is ERNIE 4.5's block with a mixture of moe_num_experts experts (mlp.gate, mlp.experts.{e}) on the layers from moe_layer_start_index to moe_layer_end_index (the last when negative) every moe_layer_interval, the others dense. Scores are a softmax; the moe_k experts are chosen with mlp.moe_statics.e_score_correction_bias added and weighted by their renormalised scores, and moe_num_shared_experts shared experts (mlp.shared_experts) run on every token. A selection bias whose length is not the expert count is refused when the layer loads.

phimoe (Phi-3.5-MoE) keeps Mixtral's tensors (block_sparse_moe.gate, experts.{e}.w1/w3/w2) under LayerNorms with bias, attention_bias on every attention projection, a biased lm_head when lm_head_bias says so, and LongRoPE whose short and long tables are multiplied by short_mscale and long_mscale. Its router is SparseMixer as Transformers runs it outside training: two rounds, each taking the highest logit not yet chosen and weighing it by its softmax over the experts whose logit lies within 2 · router_jitter_noise of that best, relative to the larger of their own magnitude and the best. A PhiMoE config with other than two experts per token, or without router_jitter_noise, is refused.

hunyuan_v1_moe (Hunyuan-A13B) is HunYuan's block with num_experts experts under mlp.gate.wg and mlp.experts.{e}, the top moe_topk weighted by their renormalised softmax, beside a shared mlp.shared_mlp num_shared_expert times intermediate_size wide unless use_mixed_mlp_moe is false; the first moe_layer_num_skipped layers stay dense. Every mixture-of-experts family may state its expert count, experts per token and expert width once or once per layer; a per-layer list whose values differ is refused, as is a HunYuan config with cross-layer attention (use_cla) or latent attention (use_mla).

telechat (TeleChat2 and TeleChat2.5) is Llama's block below transformer: word_embeddings, h.{i}, self_attention.query beside one self_attention.key_value matrix whose rows hold each head's key and then its value, self_attention.dense with a bias, mlp.gate_proj/up_proj and a biased mlp.down_proj, and ln_f; n_head, n_layer, ffn_hidden_size, layer_norm_epsilon and seq_length name the usual sizes. A config that adds the residual after the norm (apply_residual_connection_post_layernorm) or normalises the embeddings (embed_layernorm) is refused. TeleChat2's official checkpoints older than TeleChat2.5 ship as PyTorch .bin files, which Ster does not read; use a Safetensors upload. dynamic rotary scaling without an alpha is accepted as the plain rotation: it changes the base only for sequences longer than max_position_embeddings, which Ster never runs.

lfm2_moe (LFM2-MoE) is LFM2's convolution and attention stack whose first num_dense_layers layers keep the intermediate_size feed-forward and whose others route over num_experts experts (feed_forward.gate, feed_forward.experts.{e}.w1/w3/w2, moe_intermediate_size wide): sigmoid scores, the top num_experts_per_tok chosen with feed_forward.expert_bias added when use_expert_bias holds, renormalised under norm_topk_prob and scaled by routed_scaling_factor. Adapter training is refused, as on LFM2. Any selection bias whose length is not the expert count is refused when the layer loads, naming the tensor.

granitemoehybrid (Granite 4.0, the H and plain models) keeps Granite's multipliers and puts a Mamba-2 mixer (mamba, sized by Bamba's mamba_* keys) on every layer layer_types calls mamba and attention on every layer it calls attention; attention rotates only when position_embedding_type is rope (nope means no position signal). After every mixer comes either GraniteMoE's experts (block_sparse_moe) beside a shared shared_mlp whose gate and up rows share input_linear and whose down is output_linear, shared_intermediate_size wide, or, when num_local_experts is zero, that shared_mlp alone as the dense feed-forward, which takes adapters on gate, up and down. A config without layer_types or shared_intermediate_size, with a layer type other than mamba or attention, or with another position_embedding_type, is refused.

nemotron_h (Nemotron-H, Nemotron Nano 2, Nemotron 3 Nano) keeps Mamba's layout (backbone.embeddings, backbone.layers.{i}, backbone.norm_f) and gives every layer one norm and one mixer, whose kind is the layer's letter in hybrid_override_pattern: M a Mamba-2 scan (mamba_num_heads heads of mamba_head_dim, n_groups groups, ssm_state_size, conv_kernel, its gated norm taken per group), * attention with no rotation, - a feed-forward down_proj(act(up_proj(x))) with mlp_hidden_act (squared ReLU), E a mixture of n_routed_experts such feed-forwards (mixer.experts.{e}, moe_intermediate_size wide) behind DeepSeek-V3's router (mixer.gate: sigmoid scores, mixer.gate.e_score_correction_bias added to choose, n_group/topk_group limits, renormalised under norm_topk_prob, scaled by routed_scaling_factor) beside mixer.shared_experts, moe_shared_expert_intermediate_size wide, the routed experts working in moe_latent_size when it is stated. Adapter training is refused, as on every state-space family. A pattern whose length is not the layer count, or a letter other than M, *, - and E, is refused.

bailing_moe (Ling 1.x and Ling 2.0) is Llama's block under Bailing's names: model.word_embeddings, one stacked attention.query_key_value (use_qkv_bias), per-head query_layernorm and key_layernorm when use_qk_norm holds, attention.dense (use_bias), rotation over partial_rotary_factor of each head. The first first_k_dense_replace layers are dense; the rest route over num_experts experts (mlp.gate, mlp.experts.{e}) beside mlp.shared_experts, moe_intermediate_size times num_shared_experts wide. Ling 1.x scores by softmax and renormalises under norm_topk_prob; Ling 2.0's score_function sigmoid adds mlp.gate.expert_bias to choose when moe_router_enable_expert_bias holds, keeps the experts of its best topk_group of n_group groups (ranked by their best two), always renormalises and scales by routed_scaling_factor. A config with norm_head or norm_softmax, or another score_function, is refused.

TeleFLM (Tele-FLM, spelled in capitals as its config does) is Llama's block; under use_mup the embeddings are multiplied by input_mult and the logits by output_mult / mup_scale_factor, and a use_mup config without those three keys (or with a zero mup_scale_factor) is refused. The published Tele-FLM checkpoints are PyTorch .bin files, which Ster does not read; use a Safetensors conversion.

falcon_h1 (Falcon-H1) runs attention (self_attn) and a Mamba-2 scan (mamba, mamba_n_heads heads of mamba_d_head, mamba_n_groups groups, mamba_d_state, mamba_d_conv) side by side on every layer's input_layernorm output, adds both to the residual stream, then applies pre_ff_layernorm and the feed_forward block; final_layernorm closes the stack. Its muP multipliers are all applied: embedding_multiplier on the embeddings, attention_in_multiplier on attention's input and attention_out_multiplier on its output, key_multiplier on every key, ssm_in_multiplier on the scan's input, ssm_multipliers on the five parts of its projection (gate, stream, input matrix, output matrix, step), ssm_out_multiplier on its output, mlp_multipliers on the feed-forward's gate before its activation and on its output, and lm_head_multiplier on the logits. The scan's gated output passes the per-group RMS norm unless mamba_rms_norm is false. A recurrent mixer's decode state now has its own cache slot, so a layer keeps its keys and values and its scan state together. Adapter training is refused, as on every state-space family. A config with attn_layer_indices, mamba_use_mlp false, mamba_norm_before_gate, a projectors_bias unlike mamba_proj_bias, a scan width other than its heads' total, or ssm_multipliers and mlp_multipliers that are not five and two numbers, is refused.

qwen3_next (Qwen3-Next) is Qwen3-MoE's block with three changes. Every norm stores its scale as an offset from one. Its attention layers' q_proj yields each head's query rows and then as many gate rows, and attention's output is multiplied by the gate's sigmoid before o_proj. Every layer layer_types calls linear_attention (without layer_types, every layer but each full_attention_interval-th) runs gated delta-rule linear attention (linear_attn) instead: in_proj_qkvz gives each of linear_num_key_heads key heads its query and key (linear_key_head_dim wide) and the values and output gates of the linear_num_value_heads value heads it serves (linear_value_head_dim wide), in_proj_ba each value head's write strength and decay input; query, key and value pass a causal depthwise conv1d of linear_conv_kernel_dim positions and SiLU; query and key are L2-normalised and the query scaled by one over the square root of its width; each value head's state decays by exp(-exp(A_log) · softplus(a + dt_bias)) and is corrected toward the value with strength sigmoid(b); the query's read-out is RMS-normalised per head, scaled by norm, gated by the SiLU of its output gate and projected by out_proj. The experts are Qwen2-MoE's, with a shared_expert scaled by the sigmoid of shared_expert_gate. Adapter training is refused, because the linear-attention layers have no attention projection to adapt. A layer type other than these two, a missing layer_types and full_attention_interval, or a missing linear-attention size, is refused.

kimi_linear (Kimi-Linear) runs Kimi Delta Attention on the layers linear_attn_config.kda_layers names (counted from one) and DeepSeek's multi-head latent attention on the rest, unrotated when mla_use_nope holds. Kimi Delta Attention is Qwen3-Next's delta rule with separate q_proj, k_proj and v_proj, each with its own q_conv1d, k_conv1d and v_conv1d of short_conv_kernel_size positions; a decay per key channel, -exp(A_log) · softplus(f_b_proj(f_a_proj(x)) + dt_bias); write strength sigmoid(b_proj(x)); and an output o_norm gated by the sigmoid of g_b_proj(g_a_proj(x)) before o_proj, over num_heads heads of head_dim. After the first first_k_dense_replace dense layers the feed-forward is num_experts experts under block_sparse_moe (gate, experts.{e}.w1/w3/w2) beside block_sparse_moe.shared_experts, num_shared_experts times moe_intermediate_size wide, scored by moe_router_activation_func, chosen with block_sparse_moe.gate.e_score_correction_bias added (within the best topk_group of num_expert_group groups under use_grouped_topk), renormalised under moe_renormalize and scaled by routed_scaling_factor. Adapter training is refused, as on Qwen3-Next. A config without kv_lora_rank, linear_attn_config or one of its sizes, with a kda_layers entry outside the layers, or with another router activation, is refused.

Kimi-K3 (kimi_k3) keeps a kimi_linear decoder under text_config, its weights below language_model, and adds five switches. hidden_act situ multiplies the gate's beta · tanh(gate / beta) · sigmoid(gate) by the up projection's linear_beta · tanh(up / linear_beta), from activation_situ_beta (one when absent) and activation_situ_linear_beta, in the dense, shared and routed feed-forwards. linear_attn_config.use_full_rank_gate reads the delta rule's output gate from one g_proj, and gate_lower_bound bounds its log-decay as bound · sigmoid(exp(A_log) · (input + dt_bias)); an A_log stored longer than the heads (128 entries for 96) is read for its first heads, as vLLM reads it. mla_use_output_gate multiplies latent attention's output by the sigmoid of self_attn.g_proj before o_proj. routed_expert_hidden_size runs the routed experts on block_sparse_moe.routed_expert_down_proj(x) and returns their sum through routed_expert_norm (under latent_moe_use_norm) and routed_expert_up_proj; the router and shared experts read x. attn_res_block_size replaces the residual sum with attention residuals: every layer whose index the block size divides stores the running sum it received as a finished block, each sublayer reads a softmax mix of the finished blocks and the running sum scored by its self_attention_res_norm and self_attention_res_proj (or mlp_res_norm and mlp_res_proj), and model.output_attn_res_norm and model.output_attn_res_proj mix them once more before the final norm. Under attention residuals --layer reads and steers the current block's running sum. Kimi's tiktoken.model is read as a byte-level BPE with TikTokenTokenizer's split pattern and its 256 special-token ids after the ranks, those added_tokens_decoder does not name spelled <|reserved_token_{id}|>. The published Kimi-K3 weights are MXFP4 (compressed-tensors) and are refused before download; yujiepan/kimi-k3-bf16-tiny-random, random BF16 weights in the same layout, loads and generates.

inkling_text (Inkling, inside inkling_mm_model) is read as Thinking Machines stores it below model.llm: embed then its RMS norm embed_norm, layers.{i} with attn_norm and mlp_norm, the final norm, and the head unembed, which reads the final norm's output divided by logits_mup_width_multiplier and scores only the first unpadded_vocab_size tokens. Attention has no rotation: each query's score for a key carries a bias by their distance, which the query's own attn.wr_du(x) (d_rel wide per head) mixes from the layer's profiles attn.rel_logits_proj.proj [d_rel, extent], zero at a distance of extent or more. Queries and keys are RMS-normed per head (q_norm, k_norm) and their scores divided by the head width. Keys and values pass residual short convolutions (k_sconv, v_sconv: the input plus a depthwise causal convolution over conv_kernel_size positions, in F32) before the norm, and attention's and the feed-forward's outputs pass attn_sconv and mlp_sconv; a decode step convolves over the last inputs of the calls before it. The layers layer_types marks hybrid_sliding (else local_layer_ids) attend through sliding_window_size with the swa_* head shapes and that window as their extent; the hybrid layers attend fully with rel_extent, and under log_scaling_n_floor their scores and bias at position p are multiplied by 1 + log_scaling_alpha · ln(max((p + 1) / floor, 1)). The dense feed-forward stores gate and up interleaved row by row in mlp.w13_dn and multiplies its output by mlp.global_scale. The routed one (mlp_layer_types sparse, else every layer from dense_mlp_idx) scores n_routed_experts + n_shared_experts logits with mlp.gate, chooses the top num_experts_per_tok routed experts by sigmoid(logit) + mlp.gate.bias, and weighs every chosen and shared expert by its sigmoid(logit) over the sum of theirs, times route_scale and mlp.gate.global_scale; the router's weights are read on the host and carry no gradient. Thinking Machines' own config is read as Transformers reads it: dense_intermediate_size is the dense width, intermediate_size the experts', sconv_kernel_size the convolutions'. Adapters are refused. yujiepan/inkling-tiny-random, random weights in Inkling's layout, loads and generates, and a decode step continued from the cache matches the same text read whole.

minimax_m2 (MiniMax-M2) is Llama's block with full attention on every layer, query and key norms over the whole projection (qk_norm_type per_layer, or per head under per_head) unless use_qk_norm is false — Transformers' MiniMax-M2 always norms them, and its configs leave the key out — rotation over rotary_dim (or Transformers 5's rope_parameters.partial_rotary_factor) of each head, and a mixture of num_local_experts experts under block_sparse_moe (gate, experts.{e}.w1/w3/w2, intermediate_size wide) scored by scoring_func (sigmoid), chosen with block_sparse_moe.e_score_correction_bias added unless use_routing_bias is false, and renormalised. `ster parity` against Transformers on malaiwah/minimax-m2-tiny-random-bf16 measures 0.47 % relative error. MiniMax's own release is FP8 (quantization_config), which Ster refuses as every quantized checkpoint; use a BF16 conversion. A config with lightning-attention layers (attn_type_list 0) or shared experts (shared_intermediate_size above zero), or another qk_norm_type or scoring_func, is refused.

step3_text (Step3's text decoder, read from the text_config of a step3_vl checkpoint whose text weights sit at the root; the vision tower is never read) narrows each query through q_proj to share_q_dim, passes it through the RMS norm inter_norm and widens it through wq, beside num_attention_groups key-value heads (k_proj, v_proj). The layers moe_layers_enum lists (counted from zero; every layer but the first when absent) route over moe_num_experts experts whose projections are stacked under moe (gate_proj, up_proj, down_proj, one slice per expert) behind moe.gate, renormalised under norm_expert_weight, beside a shared share_expert share_expert_dim wide; the others keep the dense mlp. An adapter on query is refused, because wq reads the bottleneck rather than the hidden state. A config without share_q_dim, or with a moe_layers_enum entry that names no layer, is refused.

minimax_text_01 (MiniMax-Text-01; Transformers spells it minimax) runs lightning attention on the layers attn_type_list marks 0 (or layer_types calls linear_attention) and full attention over rotary_dim of each head on the rest. Lightning attention takes each head's query, key and value from self_attn.qkv_proj through SiLU; each head's [head_dim, head_dim] state decays by exp(-slope) per position and gains kᵀv, the slope being 2^(-8 / heads) raised to the head's number (from one) and scaled by 1 - layer / (layers - 1), as Transformers computes it; the query reads the state, and the read-out passes an RMS norm over all heads (self_attn.norm), the sigmoid of self_attn.output_gate(x) and self_attn.out_proj. Every layer ends with num_local_experts Mixtral-named experts under block_sparse_moe. Each sublayer joins the residual as residual · alpha + output · beta, the residual being the sublayer's normed input under postnorm, with the alpha and beta of layernorm_linear_attention_*, layernorm_full_attention_* and layernorm_mlp_* (Transformers' linear_attn_*_factor, full_attn_*_factor, mlp_*_factor). Adapter training is refused. A config without attn_type_list or layer_types, with another layer kind, or with shared experts, is refused.

zamba2 (Zamba2) is a stack of Mamba-2 layers (mamba, n_mamba_heads heads of mamba_headdim, mamba_ngroups groups, mamba_d_state, mamba_d_conv, the step clamped below at time_step_min as Transformers does) in which the layers layers_block_type calls hybrid also run a shared transformer block. There are num_mem_blocks such blocks, used in turn, each mapped once from the first hybrid layer that uses it (layers.{i}.shared_transformer): input_layernorm over the hidden state beside the token embeddings (attention_hidden_size wide), attention whose scores are divided by the square root of half of attention_head_dim and which rotates only under use_mem_rope, pre_ff_layernorm, and a GELU-gated feed_forward.gate_up_proj and down_proj, with no residual of its own. Each hybrid layer adds its own rank-adapter_rank terms to the block's query, key and value (use_shared_attention_adapter) and gate-and-up projection (use_shared_mlp_adapter), read from slot k of the block's adapter lists for the k-th hybrid layer; projects the block's output through its own linear; adds it to the hidden state; and runs mamba_decoder.input_layernorm and mamba_decoder.mamba, whose output joins the residual. The head is tied to the embeddings, as Transformers ties it whatever tie_word_embeddings says. Adapter training is refused, as on every state-space family. A config without layers_block_type or num_mem_blocks, with another layer kind, with adapters but no adapter_rank, or with a Mamba-2 width other than its heads' total, is refused.

Local directory#

If the --model string names an existing directory, Ster never contacts the network. It requires:

• config.json — or, for a checkpoint in Mistral's own format, params.json with no config.json beside it;
• tokenizer.json, or PLaMo's tokenizer.jsonl when there is no tokenizer.json, or else a tokenizer.model that its tokenizer_config.json names ChatGLM4Tokenizer, or else Kimi's tiktoken.model that it names TikTokenTokenizer (any other tokenizer.model is refused by its class);
• at least one top-level file whose extension is .safetensors and whose file name contains neither optimizer nor training_args: the consolidated* files for Mistral's format, every other one for Transformers'; or, with no such file, exactly one .gguf file — a llama.cpp fine-tune laid beside its base model's config.json and tokenizer.

Weight names are sorted before memory mapping. A sharded checkpoint works when all shards are top-level Safetensors files; the index JSON is not consulted. --revision, when supplied for a local model, is metadata only: it is recorded verbatim in output artifacts and does not select files. Without it, model_revision is null.

A GGUF directory is how a fine-tune llama.cpp published runs on Ster's own decoder: each tensor is read out of the file under llama.cpp's name (token_embd, blk.{i}.attn_q, ffn_gate, output_norm), dequantized at --precision, and given to the decoder under its Transformers name, so generate, decide, steering and capture work on it as on safetensors. Example, Jeden's goal model: download the base model's config.json, tokenizer.json and tokenizer_config.json and the goal model's .gguf file into one directory, then run ster generate --model <dir> --system goal-system-prompt.md --prompt <text> --precision bf16 --device metal with the token budget, temperature and seed the caller states. Two .gguf files in one directory are refused with their count; a name the file does not hold is refused naming the GGUF tensor and the name it was asked for; a tensor whose shape differs from the config's is refused with both shapes; ster tune merge refuses a GGUF base, because a merge writes the base's tensors back and a GGUF's are quantized.

A nonexistent directory-looking string is not treated as a missing local path: it falls through and is treated as a Hub repository id. Use an existing path (or fix the path) to force local resolution.

Hugging Face Hub#

Anything that is not an existing directory is a Hub model id. Ster requests the repository at --revision, defaulting to main; downloads config.json (or params.json when the repository publishes only Mistral's own format, as Mistral Large 3 does) and tokenizer.json; discovers every published .safetensors sibling of that format — consolidated* for Mistral's, every other one for Transformers' — excluding names containing optimizer or training_args; and downloads each weight file. A repository that publishes both formats (Mistral 7B v0.3) is read as Transformers', so its weights are downloaded and mapped once. A Mistral-format params.json is put in the Transformers shape as vLLM reads it — Mistral Large 3 (latent attention, experts beside a shared one) becomes deepseek_v3 with softmax scores renormalised and Llama 4's query temperature from llama_4_scaling, its yarn block a YaRN rotation; a dense model becomes mistral rotating adjacent pairs — and every tensor is read under Mistral's name (tok_embeddings, layers.{i}.attention.wq_a, feed_forward.w1, experts.{e}.w1, output); its end-of-sequence token is the tokenizer config's eos_token. Experts without a shared one (Mixtral's layout) in Mistral's format are refused. ster tune merge writes a Mistral-format source back as consolidated.safetensors beside its params.json. The repository's resolved commit SHA, not the branch name, becomes model_revision in artifacts.

The architecture gate and the quantized-checkpoint refusal run on the downloaded config, before any weight file is fetched: a checkpoint Ster cannot run costs its config and tokenizer, not every shard. pfnet/plamo-2-1b (model_type plamo2) is refused after 3.2 MB.

Ster calls hf_hub::api::sync::Api::new, whose dependency version uses ~/.cache/huggingface/hub and reads an optional token from ~/.cache/huggingface/token. This constructor does not use hf-hub's from_env path, so HF_HOME and HF_ENDPOINT do not configure Ster 0.13. There is no Ster-specific token or offline flag. For a deterministic offline run, pass a local directory.

Architecture gate#

The config is first parsed as JSON and its top-level model_type must be one of the families above. Missing, null, or any other value is refused before weights are mapped:

model architecture "plamo2" is unsupported by this Ster build; use a Hugging Face checkpoint whose model_type is one of llama, mistral, mixtral, qwen2, qwen2_moe, qwen3, qwen3_moe, phi, phi3, granite, granitemoe, stablelm, starcoder2, cohere, cohere2, nemotron, olmo, olmo2, olmo3, olmoe, exaone4, internlm3, seed_oss, arcee, ernie4_5, minicpm, orion, glm, glm4, gpt_neox, gptj, smollm3, gpt2, gpt_bigcode, opt, bloom, falcon, mpt, deepseek_v2, deepseek_v3, minicpm3, mamba, falcon_mamba, glm4_moe, internlm2, exaone, jamba, hunyuan_v1_dense, mamba2, bamba, gpt_oss, lfm2, ernie4_5_moe, dbrx, phimoe, hunyuan_v1_moe, telechat, lfm2_moe, granitemoehybrid, nemotron_h, jais2, bailing_moe, TeleFLM, falcon_h1, qwen3_next, kimi_linear, minimax_m2, step3_text, minimax_text_01, minimax, zamba2, gemma, gemma2, gemma3_text, gemma4_text, gemma4_unified_text, mimo, mellum, flex_olmo, granite_swa, granitemoe_swa, granitemoeshared, glm4_moe_lite, iquestcoder, hyperclovax, vaultgemma, apertus, exaone_moe, solar, step1, telechat3, param2moe, PanguEmbedded, olmo_hybrid, hy_v3, nanbeige, iquestloopcoder, nemotron-nas, deepseek_v32, glm_moe_dsa, axk1, bailing_hybrid, mimo_v2_flash, mimo_v2, step3p5, k2_horizon, pangu_ultra_moe, longcat_flash, sarvam_moe, sarvam_mla, cohere2_moe, deepseek, ministral3, llama4_text, qwen3_5_text, qwen3_5_moe_text, afmoe, nemotron_h_puzzle, chatglm, laguna, longcat_flash_ngram, muse_glimmer_text, plamo3, gemma3n_text, hrm_text, inkling_text, deepseek_v4, glm5_next_text, hy_v4, minimax_m3_vl_text, dots3_note

The same bytes must then deserialize as Candle's LlamaConfig, after Ster reads a Transformers 5 rope_parameters as rope_theta and rope_scaling, takes out a linear, longrope or yarn rope_scaling it applies itself (HunYuan's alpha-scaled dynamic becomes a fixed rope_theta; plain dynamic is the unscaled rotation within max_position_embeddings) and copies each family's own spelling of a dimension under Llama's name; Llama 3's scaling stays in Candle's config. Ster builds its own decoder loop and always loads and computes weights as F32 unless --precision says otherwise. Quantized formats and non-Safetensors weights are not resolution paths.

Device choice#

cpu is always available. metal and cuda are compile-time Cargo features and select device 0; passing either to a binary built without its feature is a refusal, not a CPU fallback. Build with --features metal or --features cuda and then pass the matching --device value.

Refusals#

Local and shared load failures begin with:

• failed to list <DIR>
• model config is missing: <PATH>
• tokenizer is missing: <PATH>
• model directory contains no safetensors weights
• failed to read <CONFIG_PATH>
• invalid model config <CONFIG_PATH>
• invalid Llama config <CONFIG_PATH>
• failed to load tokenizer <PATH>: <DETAIL>
• failed to map <N> model weight files

Hub setup adds:

• failed to initialize Hugging Face Hub client
• failed to read model repository <MODEL>
• model <MODEL> publishes no safetensors weights
• failed to download <WEIGHT_NAME>

Device failures and operational fixes are collected in the runbook.