everything in the runtime
One Rust core on top of llama.cpp, exposed six ways, wire-compatible with the tools you already use, and deployable from a laptop to a GPU box.
One core, six idiomatic APIs
The native core. A safe Rust API over llama.cpp — Model, Context, ContextParams.
pip install mullama. Idiomatic Python, not transliterated Rust; supports n_gpu_layers and n_ctx.
npm install mullama. A native addon, not an HTTP client — the model lives in your Node process.
go get github.com/cognisoc/mullama. In-process inference with ModelParams / ContextParams.
composer require mullama/mullama. Load a model and generate from PHP without a sidecar daemon.
Link the static C ABI library directly — mullama_load_model, mullama_new_context, mullama_generate.
Drop-in where it counts
Same verbs: run, pull, serve, chat, list, ps, create, show, rm, cp. Muscle memory transfers.
The same Modelfile format and model registry — existing Modelfiles work unchanged.
Serves on the Ollama port, so existing client code pointed at localhost:11434 keeps working.
OpenAI-compatible surface at /v1 — the openai SDK, LangChain, and LlamaIndex just work.
Anthropic-compatible at /v1/messages — the anthropic SDK works too. Ollama has no equivalent.
A built-in web UI and terminal UI ship with the runtime.
Runs where you run
Link the static C ABI into a desktop, CLI, edge, or mobile build — no daemon, no network.
CUDA, Metal, ROCm, Vulkan, OpenCL, SYCL, and RPC for distributed inference. CPU out of the box.
Metal is selected automatically on Apple Silicon; no configuration required.
ghcr.io/cognisoc/mullama for CPU and :<ver>-cuda for NVIDIA.
Per-model resource limits, LRU model eviction, persistent stats via sled, Prometheus metrics.
A shell one-liner pulls a static binary for Linux and macOS.
What the runtime can do
Any GGUF file from Hugging Face, with aliases for Llama, Qwen, DeepSeek, Mistral, Phi, Gemma.
Vision via LLaVA, llava-phi3, and Moondream; the model resolves from a short alias.
Real-time audio capture with voice activity detection and streaming responses.
Built-in embeddings via BGE and Nomic for RAG pipelines.
Grammar-constrained JSON output for structured generation.
Work-stealing scheduling for parallel inference across documents (parallel feature).