weicj / vLLM-2080Ti-Definitive
The definitive vLLM runtime for dual RTX 2080 Ti 22GB + NVLink, delivering Qwen 27B local inference with maximum 200+ tok/s single-request decode with support of FP8 weight ( Join Discord :https://discord.gg/VFqVVySdMS )
- Repository stars
- 1,206All-time #36,015
- Fork
- 168
- Last push
- updated 1 months ago
- Repository created
- since 2026
About
The definitive vLLM runtime for dual RTX 2080 Ti 22GB + NVLink, delivering Qwen 27B local inference with maximum 200+ tok/s single-request decode with support of FP8 weight ( Join Discord :https://discord.gg/VFqVVySdMS )Summarised by AI from the README; the original prevails
Author description: The definitive vLLM runtime for dual RTX 2080 Ti 22GB + NVLink, delivering Qwen 27B local inference with maximum 200+ tok/s single-request decode with support of FP8 weight ( Join Discord :discord.gg/VFqVVySdMS )
Basic listing based on official public information; functionality has not been independently evaluated.
- Pricing
- Pricing not verifiedCheck current official software, hosting, model and infrastructure pricing.