/use-cases

what you can build

Mullama is one runtime for the local, embedded, and single-box case — from a desktop app to a production API server. Here is where it earns its place.

Polyglot teams, one dependency

A Node frontend, a Python data pipeline, and a Go control plane can all call the same inference runtime through native, in-process bindings — no subprocess shims, no duplicated HTTP clients, no FFI you wrote at 2am.

›Native bindings for Rust, Python, Node, Go, PHP, C
›One core, six idiomatic APIs
›No daemon required between app and model

Desktop, CLI & mobile apps

Link the static C ABI library directly into an Electron, Tauri, or native iOS/Android build and ship inference inside the app binary. The model runs in-process with no network dependency.

›Static C ABI you link at build time
›No orchestration, no background service
›Works offline by construction

Drop-in Ollama migration

Keep the CLI, Modelfiles, model registry, and port 11434 you already use. Point existing client code at Mullama unchanged, then adopt native bindings or the Anthropic-compatible API when you are ready.

›Same verbs, Modelfile, and port
›OpenAI + Anthropic HTTP APIs
›Adopt bindings incrementally

Production API servers

Serve an OpenAI-compatible API with SSE streaming, per-model resource limits, LRU model eviction, and Prometheus metrics. LangChain, LlamaIndex, and the official SDKs all work pointed at localhost.

›OpenAI + Anthropic wire compatibility
›Per-model limits and LRU eviction
›Prometheus metrics and persistent stats

RAG & structured output

Built-in embeddings (BGE, Nomic) and grammar-constrained JSON output make retrieval pipelines and structured extraction a first-party concern rather than a bolt-on.

›Embeddings via BGE and Nomic
›Grammar-constrained JSON output
›ColBERT-style late interaction

Edge & on-prem inference

For teams that cannot or will not send prompts to a hosted API, Mullama embeds a quantized GGUF model on the box. Nothing leaves the machine unless you explicitly point it elsewhere.

›Quantized GGUF models on-device
›7 GPU backends, CPU out of the box
›No hosted service, no telemetry

// where mullama is not the tool

Mullama runs GGUF models locally; it is not a model trainer, not a SafeTensors runtime, and not a hosted service. If you need large-scale hosted serving with continuous batching, use vLLM. See the comparisons for the full picture.