mlc-ai / tokenizers-cpp
Universal cross-platform tokenizers binding to HF and sentencepiece
- Repository stars
- 513All-time #60,855
- Fork
- 131
- Last push
- updated 5 months ago
- Repository created
- since 2023
About
Universal cross-platform tokenizers binding to HF and sentencepieceSummarised by AI from the README; the original prevails
Author description: Universal cross-platform tokenizers binding to HF and sentencepiece
Basic listing based on official public information; functionality has not been independently evaluated.
- Pricing
- Pricing not verifiedCheck current official software, hosting, model and infrastructure pricing.