microsoft / LLMLingua
[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
- Repository stars
- 6,806All-time #8,499
- Fork
- 451
- Last push
- updated 1 months ago
- Repository created
- since 2023
About
Summarised by AI from the README; the original prevails
Author description: [EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
Basic listing based on official public information; functionality has not been independently evaluated.
Check before adopting
- No pushes for over 6 months (last 2026-09-10); check whether it is maintained.
- Platform
- Not verified
- Pricing
- Pricing not verifiedCheck current official software, hosting, model and infrastructure pricing.
- Maintenance notes
- Not verified