jmaczan / tiny-vllm
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM
- Repository stars
- 1,182All-time #36,465
- Fork
- 102
- Last push
- updated 5 days ago
- Repository created
- since 2026
About
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLMSummarised by AI from the README; the original prevails
Author description: Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM
Basic listing based on official public information; functionality has not been independently evaluated.
- Pricing
- Pricing not verifiedCheck current official software, hosting, model and infrastructure pricing.