Evaluation tools
25 repositories · updated 2026-11-06
2sharkdp / hyperfineA command-line tool for benchmarking command execution.RustApache-2.0 · PermissiveInsufficient rank history29,218
3confident-ai / deepevalThe LLM Evaluation FrameworkPythonApache-2.0 · PermissiveInsufficient rank history19,266
4raga-ai-hub / RagaAI-CatalystPython SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph viewPythonApache-2.0 · PermissiveInsufficient rank history16,186
5EleutherAI / lm-evaluation-harnessRun language-model tasks and aggregate results through a shared evaluation framework.PythonMIT · PermissiveInsufficient rank history14,413
6dotnet / BenchmarkDotNetA library for benchmarking .NET code.C#MIT · PermissiveInsufficient rank history11,526
7google / benchmarkA library for writing and running microbenchmarks.C++Apache-2.0 · PermissiveInsufficient rank history10,500
8mcollina / autocannonfast HTTP/1.1 benchmarking tool written in Node.jsJavaScriptMIT · PermissiveInsufficient rank history8,534
9open-compass / opencompassManage language-model evaluations, dataset configurations and result summaries.PythonApache-2.0 · PermissiveInsufficient rank history7,566
10codesenberg / bombardierFast cross-platform HTTP benchmarking tool written in GoGoMIT · PermissiveInsufficient rank history6,854
11evcxr / evcxrInteractive Rust evaluation tools, including a Jupyter kernel, a Rust REPL and related shared libraries.RustLicense not declaredInsufficient rank history6,529
12erikbern / ann-benchmarksA Python benchmark project comparing approximate-nearest-neighbor libraries.PythonMIT · PermissiveInsufficient rank history5,750
13bheisler / criterion.rsA Rust benchmarking library using statistical analysis.RustApache-2.0 · PermissiveInsufficient rank history5,548
14six-ddc / plowA high-performance HTTP benchmarking tool that includes a real-time web UI and terminal displayGoApache-2.0 · PermissiveInsufficient rank history4,515
15EvolvingLMMs-Lab / lmms-evalA multimodal evaluation toolkit covering text, image, video, and audio tasks.PythonLicense not declaredInsufficient rank history4,485
16open-compass / VLMEvalKitOpen-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarksPythonApache-2.0 · PermissiveInsufficient rank history4,474
17devMEremenko / XcodeBenchmarkA benchmark measuring large-project compilation times in Xcode on Mac hardware.PythonMIT · PermissiveInsufficient rank history3,688
18OpenDCAI / DataFlex[NeurIPS 2026] DataFlex: A Unified Benchmark and Evaluation Platform for Data-Centric Training of Large Language ModelsPythonApache-2.0 · PermissiveInsufficient rank history3,432
19ianarawjo / ChainForgeAn open-source visual programming environment for battle-testing prompts to LLMs.TypeScriptMIT · PermissiveInsufficient rank history3,040
20processone / tsungTsung is a high-performance benchmark framework for various protocols including HTTP, XMPP, LDAP, etc.ErlangGPL-2.0 · Share derivative sourceInsufficient rank history2,630
