GitHub stars · StarDay

1mozilla-ai / llamafileDistribute and run LLMs with a single file.C++License not declared26,574

2altic-dev / FluidVoiceAuthor description: Fastest and only macOS Dictation app with on-device STT and custom trained AI enhancement model. Windows, iOS testflight! Android, Linux waitlist open! A local Wispr Flow alternative. Follow https://x.com/fluidvoiceapp for moreSwiftGPL-3.0 · Share derivative source12,856

3getumbrel / llama-gptAuthor description: A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!TypeScriptMIT · Permissive10,923

4QwenAudio / SenseVoiceOpen-source SenseVoiceSmall model for Mandarin, Cantonese, English, Japanese, and Korean ASR, language ID, emotion recognition, and audio event detection.CMIT · Permissive9,702

5xorbitsai / inferenceAuthor description: Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.PythonApache-2.0 · Permissive9,639

6Osmantic / ODSODS V3 Pre-Release: Public testing and refinement ahead of the official V3 launch. Turn your PC, Mac, or Linux box into a private AI server.PythonApache-2.0 · Permissive7,780

7ARahim3 / DigUpSearch your computer by what's in your files: photos, PDFs, documents, recordings, videos and code. On-device on Mac, Windows and Linux, built on Google DeepMind's EmbeddingGemma 2.SwiftMIT · Permissive6,275

8SciSharp / LLamaSharpA C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.C#MIT · Permissive3,826

9nicedreamzapp / claude-code-localHit your Claude Code usage limit or ran out of credits? Keep working on a local model. Claude Code on your own Mac (MLX), Linux or Windows (llama.cpp) with no API key and no rate limits. Your code stays home.PythonMIT · Permissive3,475

10AtomicBot-ai / atomic-agentAtomic Agent is a local-first AI agent. Runs open-weight models on your own machine via llama.cpp.TypeScriptMIT · Permissive3,398

11docker / compose-for-agentsBuild and run AI agents using Docker Compose. A collection of ready-to-use examples for orchestrating open-source LLMs, tools, and agent runtimes.TypeScriptApache-2.0 · Permissive1,070

12defilantech / LLMKubeKubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.GoApache-2.0 · Permissive251