benbrandt / text-splitter
Split text into semantic chunks, up to a desired chunk size. Supports calculating length by characters and tokens, and is callable from Rust and Python.
- Repository stars
- 645All-time #52,796
- Fork
- 35
- Last push
- updated 1 days ago
- Repository created
- since 2023
About
Split text into semantic chunks, up to a desired chunk size. Supports calculating length by characters and tokens, and is callable from Rust and Python.Summarised by AI from the README; the original prevails
Author description: Split text into semantic chunks, up to a desired chunk size. Supports calculating length by characters and tokens, and is callable from Rust and Python.
Basic listing based on official public information; functionality has not been independently evaluated.
- Pricing
- Pricing not verifiedCheck current official software, hosting, model and infrastructure pricing.