/how-it-works

a runtime layer, not a reimplementation

Mullama sits on top of llama.cpp and adds the layer everything else needs: one Rust core, six bindings, an Ollama-compatible daemon, and two HTTP APIs — all sharing a single implementation.

// the stack

six layers, one core

llama.cpp engine

The inference engine underneath. Mullama does not reimplement inference — it builds a runtime layer on top of llama.cpp and its GGUF support and GPU backends.

Rust core

A safe Rust API — Model, Context, ContextParams — over the engine. This is the single implementation every binding and every server surface shares.

Language bindings

Idiomatic in-process bindings for Rust, Python, Node.js, Go, PHP, and C/C++ generated over the same core, so behaviour is identical across languages.

Daemon & CLI

An Ollama-compatible daemon and CLI (run, pull, serve, chat, list, ps, create, show, rm, cp) with the same Modelfile format, registry, and port 11434.

HTTP surface

OpenAI-compatible /v1 and Anthropic-compatible /v1/messages endpoints, served by the daemon for clients that expect a network API.

Model registry

Resolves short aliases like llama3.2:1b to GGUF files on Hugging Face, with hf:owner/repo and local .gguf escape hatches.

// request flow

from call to tokens

01 You call generate() from a binding, or send a request to the OpenAI/Anthropic endpoint.
02 The model registry resolves the alias to a GGUF file (pulling from Hugging Face on first use).
03 The Rust core loads the model through llama.cpp on the selected GPU backend, or CPU.
04 A Context holds the KV state; tokens are produced and streamed back.
05 In library mode the tokens return in-process; in server mode they stream over SSE.

try it → vs Ollama vs llama.cpp