GitHub stars · StarDay

1modelscope / FunASROpen-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.PythonMIT · Permissive20,954

2meta-llama / llama-cookbookWelcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We also show you how to solve end to end problems using Llama model family and using them on various provider servicesJupyter NotebookMIT · Permissive18,582

3budtmo / docker-androidAndroid in docker solution with noVNC supported, video recording, mcp server and AI-agentPythonLicense not declared16,098

4Orchestra-Research / AI-Research-SKILLsComprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent with full horsepower. Maintained by Orchestra Research.TeXMIT · Permissive14,319

5halfrost / Halfrost-Field✍🏻 Source Code Deep Dives, System Design & Engineering Blogs | Halfrost-Field Frost Land: Source code analysis, system design and engineering practice notesGoCC-BY-SA-4.0 · Share derivative source13,251

6LMCache / LMCacheAuthor description: LMCache: Supercharge Your LLM with the Fastest KV Cache LayerPythonApache-2.0 · Permissive12,302

7OpenRLHF / OpenRLHFAn Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL)PythonApache-2.0 · Permissive10,143

8xorbitsai / inferenceAuthor description: Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.PythonApache-2.0 · Permissive9,646

9ai-dynamo / dynamoA Datacenter Scale Distributed Inference Serving FrameworkRustLicense not declared8,451

10kvcache-ai / MooncakeMooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.C++Apache-2.0 · Permissive6,996

11vllm-project / semantic-routerAn open, programmable decision layer for models and compute.GoApache-2.0 · Permissive6,413

12mostlygeek / llama-swapReliable model swapping for any local AI server. Chosen by thousands of users. Batteries included. If we don't have it, you probably don't need it.GoMIT · Permissive6,204

13kserve / kserveStandardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on KubernetesGoApache-2.0 · Permissive6,197

14gpustack / gpustackA GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.PythonApache-2.0 · Permissive5,984

15OpenBMB / UltraRAGA Low-Code MCP Framework for Building Complex and Innovative RAG PipelinesPythonApache-2.0 · Permissive5,738

16skyzh / tiny-llmlearn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + QwenPythonApache-2.0 · Permissive5,678

17xlite-dev / Awesome-LLM-Inference📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉PythonGPL-3.0 · Share derivative source5,548

18katanaml / sparrowStructured data extraction, instruction calling and agentic workflows with ML, LLM and Vision LLMPythonGPL-3.0 · Share derivative source5,232

19lemony-ai / cascadeflowCascading runtime for AI agents. Optimize cost, latency, quality, and policy decisions inside the agent loop.PythonMIT · Permissive3,925

20PaddlePaddle / FastDeployHigh-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddlePythonApache-2.0 · Permissive3,725