All projects
6 repositories · updated 2026-11-07
Loading…
1xorbitsai / inferenceAuthor description: Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.PythonApache-2.0 · PermissiveInsufficient rank history9,646
2ai-dynamo / dynamoA Datacenter Scale Distributed Inference Serving FrameworkRustLicense not declaredInsufficient rank history8,451
3kvcache-ai / MooncakeMooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.C++Apache-2.0 · PermissiveInsufficient rank history6,996
4gpustack / gpustackA GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.PythonApache-2.0 · PermissiveInsufficient rank history5,984
5Gen-Verse / OpenClaw-RLOpenClaw-RL: Train any agent simply by talkingPythonApache-2.0 · PermissiveInsufficient rank history5,743
6patchy631 / time-to-first-tokenA 10-week, 30-minutes-a-day roadmap for LLM inference serving and optimization. vLLM, SGLang, quantization, speculative decoding, benchmarking.HTMLApache-2.0 · PermissiveInsufficient rank history1,032
