Open source · Rust
Rich Triplet
A complete LLM stack, built from scratch in Rust — without a single ML dependency. Gemma 3 inference on Apple Silicon, Metal GPU decode at 17+ tokens/s in ~6 GB of RAM.
- Rust
- No ML dependencies
- Apple Silicon · Metal
- 412 tests
$ rich-triplet --model gemma3-4b \
--prompt "What is the capital of France?" --temp 0.8
→ loading gemma-3-4b-it-q4_0.gguf · 34 layers · Metal
The capital of France is Paris.
▸ 17.3 tok/s · ~6 GB RAM
What’s inside
Every component hand-written, from tensors to GPU kernels. No black boxes, no magic.
No ML dependencies
Matrix math, autograd, attention, quantization and tokenizer — all hand-written. Not a single machine-learning library.
Gemma 3 inference
Runs Gemma 3 4B from Q4_K_M / Q4_0 GGUF weights: CPU prefill, GPU decode, KV cache in unified memory.
Full-graph Metal GPU
The whole forward pass encoded as ~717 dispatches inside a single Metal command buffer. 17.3 tokens/s on Apple Silicon.
Three transformers
GPT-2, GPT-OSS (RoPE, GQA, Mixture of Experts) and Gemma 3 (sliding window, four-norm blocks).
Autograd engine
An automatic-differentiation engine built from scratch: the chain rule, matrix VJPs, Q4K/BF16 GEMV, ARM NEON + SDOT.
412 tests
Correctness verified: finite-difference gradients, attention shapes, quantization, Metal kernels and tokenizer.
Open source
Read the code
It’s all in the repo: 12+ Rust modules, from tensors to Metal kernels, with benchmarks and 412 tests. Clone it, read it, run it.
Apple Silicon recommended for Metal GPU decode.