EleutherAI / lm-evaluation-harness
Run language-model tasks and aggregate results through a shared evaluation framework.
- Repository stars
- 14,413All-time #3,468
- Fork
- 3,772
- Last push
- updated 2 days ago
- Repository created
- since 2020
About
Summarised by AI from the README; the original prevails
Install the harness and desired model backend, then select tasks and run a configuration.
Use cases：For comparing language models with reusable task definitions. Selection note: Requires model access and compute; results depend on tasks, prompts and runtime configuration.
Deployment：Install the harness and desired model backend, then select tasks and run a configuration.
- Platform
- Not verified
- Pricing
- Pricing not verifiedCheck hosted-service, model and infrastructure costs for the option you choose.