Choose a local model to download. F32 is higher quality. Q8 is smaller and runs better on mobile, but it will often appear dumber because quantization reduces weight precision. Only one model is cached at a time.
To switch models later, open ? and use Reset to clear the local cache.
Gab is a small language model ~100 million parameters. It runs locally inside your browser, nothing you type leaves your machine.
Choose either the full F32 weights, ~381 MiB, or the smaller Q8 weights, ~96 MiB. F32 is higher quality. Q8 is faster to download. The selected model is cached, subsequent page loads won't need to download.
Model: Gab100M on Hugging Face
Training data: Gab100MPretrain and Cosmopedia v2
Fine-tuning data: Gab100MFinetune and SmolTalk
Blog/site: gabormakesgames.com
This is a research project running a small local model. Refresh to start over. There's no chat history kept between sessions.
Decoder-only transformer with pre-normalization, rotary position embeddings, exact GeLU feed-forward blocks with a hidden bias, and tied input/output embeddings.
For token ids x, look up embeddings h = E[x], where E has shape [10000, 768]. There is no position embedding table; position enters only through RoPE inside attention. Each of the 13 blocks applies:
h = h + Attention(RMSNorm(h))
h = h + MLP(RMSNorm(h))
After the final block:
logits = RMSNorm(h) @ E.T
There is no separate output head; logits use the tied embedding matrix.
Each norm has one learned scale vector g of length 768 and no bias:
RMSNorm(x) = g * x / sqrt(mean(x^2) + 1e-5)
The model is trained per head: each of the 12 heads owns its own
Wq, Wk, Wv of shape [768, 64] and its
own output projection Wo of shape [64, 768]. Heads are never
concatenated — each head projects its result back up to 768 on its own and the
per-head results are summed:
q_i = x @ Wq_i, k_i = x @ Wk_i, v_i = x @ Wv_i
scores_i = (q_i @ k_i.T) / sqrt(64)
attn_i = softmax(causal_mask(scores_i)) @ v_i
Attention(x) = sum_i(attn_i @ Wo_i)
Summing per-head outputs is algebraically identical to a concat plus one big
[768, 768] output projection, so this runtime fuses the per-head
matrices at export time and runs the standard fused formulation — the results
are bit-identical. All projections are bias-free.
Rotary position embeddings use interleaved rotation: adjacent channels (2i, 2i+1) form rotation pairs (unlike Llama-style split-half RoPE). For each pair index i in a 64-wide head:
freq_i = 1 / theta^(2i / 64), with theta = 10000
angle = position * freq_i
[a, b] -> [a*cos(angle) - b*sin(angle), a*sin(angle) + b*cos(angle)]
The feed-forward path is a two-matrix GeLU MLP — no gate, but with a learned per-neuron bias on the hidden layer:
u = x @ Wup + b, where Wup is [768, 3072] and b is [3072]
m = GeLU(u)
MLP(x) = m @ Wdown, where Wdown is [3072, 768]
GeLU is the exact form:
GeLU(x) = 0.5 * x * (1 + erf(x / sqrt(2)))
The tokenizer is byte-level BPE with 10,000 ids: ids 0-255 are the raw bytes and
every other id is a merge of two earlier ids, learned in order. Encoding applies
every merge rule in sequence to the UTF-8 bytes; decoding unrolls an id back down
to bytes. Reserved tokens — <|user|>, <|assistant|>,
<|end|>, <|endoftext|>, <think>,
</think> — are chained merges taught before training, so they
encode to a single id and are atomic.
Inference runs on a pure-JavaScript CPU runtime with a float32 KV cache — the
model is small enough that no GPU is needed. Weights live in
IndexedDB; Q8 keeps matrix weights quantized in memory for
smaller/faster mobile runs, while F32 keeps the original fine-tuned precision.
Q8 uses symmetric rowwise int8 weights with float32 scales (norms and the MLP
bias stay float32):
weight[row, col] = int8[row, col] * scale[row]
Turns are concatenated with no whitespace or separators between special tokens:
<|user|>…<|end|><|assistant|><think>…</think>…<|end|>
Thinking is optional and lives inside the assistant turn. Turn off Chat view to see the actual token stream as the model sees it, sub-word BPE tokens included.
This clears the locally cached model files from IndexedDB. Use it if the download was interrupted, the model files changed, or you want to switch between Q8 and F32.
Your current chat will be lost and the page will reload. After reload, the app will show the download screen again.