ollama-alternative

Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4× faster than Apple's MLX (mlx-lm) on the same weights.

llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.

Local AI desktop app: a Chromium browser with a GPU-aware llama.cpp runtime, OpenAI-compatible API, AI chat and agents, Stable Diffusion image/video, TTS and music, plus an MCP server so Claude Code, Codex and Cline can drive your browser. Windows, macOS, Linux.

Open-source control panel for the real llama.cpp: see where a model lands in your VRAM before loading, every llama-server flag per model, one-click llama.cpp updates, several models at once, and Embers (local agents whose quotes are checked by code). Windows · Linux · macOS.

A private Claude-Code-style coding agent for Apple Silicon — run chat, code, and local model workflows on-device. MLX-native, Ollama/OpenAI API compatible, zero API keys.

The local AI app built for the Mac. Run LLMs like Qwen, Gemma and GPT-OSS on Apple Silicon, including models far bigger than your Mac's memory, by streaming MoE experts from the SSD.

Free, open-source app that runs AI chat models on your own computer — and lets computers team up over the internet to run models too big for one machine. No account, no crypto. OpenAI- and Anthropic-compatible API.

🔥 Alternative to Ollama — multi-model serving with sub-ms model switching · CPU-only 20B inference for Edge AI · llama.cpp + stablediffusion.cpp

Ollama-compatible LLM server on a tuned llama.cpp fork. Beats vllm by 40% on single stream decode, and by 20% on multi stream decode, and beats ollama by much 2-4x. Made possible with custom CUDA kernels, MTP, speculation and its own 3-bit builds. Uncensors models in one command. Windows and Linux, NVIDIA. Optimized to not eat up ram.

Pure-Go, no-cgo local LLM inference — run Gemma, Qwen, Llama and friends from safetensors or GGUF in a single static binary.

Local AI platform: run LLMs, Stable Diffusion, Whisper, TTS, video & 3D generation offline — open-source Ollama + ComfyUI alternative in one binary.

⚡️ The fastest way to run local LLMs on Apple Silicon — sub-second model loads, beats Ollama on throughput, tail latency, and full-response time. OpenAI/Ollama-compatible. No cloud, no API keys.

Run open AI models on your own computer — a friendly desktop app and local gateway for llama.cpp and more. Private, free, open source.

caix — native Apple Core AI inference server for Apple silicon (beta): OpenAI/Anthropic API, dashboard, streaming chat with tools/skills/MCP, MTP speculative decoding.

The Ollama developer experience with the vLLM production power. Deploy local LLMs via Docker with smart and automatic GPU VRAM management.

Sparsify by Daylink — run MoE models bigger than your RAM: expert weights page from SSD on demand, byte-identical output. Mixtral 26GB in 3.3GB RSS on a MacBook Air.

Archive of every local LLM tested on one NVIDIA RTX 2080 8GB (TU104) on Linux — recipes, runtimes, quants, pinned sources, settings, context ceilings, and reproducible measurements. Failures included.

local LLM gateway (Go) OpenAI-compatible with real failover, multi-account aliasing, and ChatGPT Plus/Pro as a backend. Single ~11MB binary, no Docker, secrets in OS keyring. Windows/macOS/Linux.

Web frontend for LM Studio — browser access, adaptive memory, multi-user auth, MCP tools

Local LLM gateway for Apple Silicon. Works with OpenClaw, Hermes Agent, Claude Code, and Codex (AIRun). No Ollama or LM Studio required.