GitHub stars · StarDay

1wg / wrkModern HTTP benchmarking toolCLicense not declared40,427

2langfuse / langfuse🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.TypeScriptLicense not declared37,105

3Tencent / WeKnoraOpen-source LLM knowledge platform: turn raw documents into a queryable RAG, an autonomous reasoning agent, and a self-maintaining Wiki.GoLicense not declared36,659

4mastra-ai / mastraMastra is the modern TypeScript framework for AI-powered applications and agents.TypeScriptLicense not declared29,384

5mlflow / mlflowThe open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.PythonApache-2.0 · Permissive28,698

6promptfoo / promptfooTest your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.TypeScriptMIT · Permissive26,896

7comet-ml / opikAuthor description: Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.PythonApache-2.0 · Permissive23,223

8confident-ai / deepevalThe LLM Evaluation FrameworkPythonApache-2.0 · Permissive19,337

9vibrantlabsai / ragasSupercharge Your LLM Application Evaluations 🚀PythonApache-2.0 · Permissive16,249

10raga-ai-hub / RagaAI-CatalystPython SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph viewPythonApache-2.0 · Permissive16,186

11EleutherAI / lm-evaluation-harnessRun language-model tasks and aggregate results through a shared evaluation framework.PythonMIT · Permissive14,413

12heroiclabs / nakamaScalable open-source game backend server: multiplayer, matchmaking, leaderboards, chat, and social features for games.GoApache-2.0 · Permissive13,682

13Arize-ai / phoenixAI Observability & EvaluationPythonLicense not declared12,172

14mrgloom / awesome-semantic-segmentation:metal: awesome-semantic-segmentationLicense not declared10,849

15mcollina / autocannonfast HTTP/1.1 benchmarking tool written in Node.jsJavaScriptMIT · Permissive8,534

16FailproofAI / failproofaiObservability and enforcement for AI agent harnesses. Capture every run and runtime reliability with policy enforcement.TypeScriptLicense not declared8,116

17open-compass / opencompassManage language-model evaluations, dataset configurations and result summaries.PythonApache-2.0 · Permissive7,566

18codesenberg / bombardierFast cross-platform HTTP benchmarking tool written in GoGoMIT · Permissive6,854

19harbor-framework / harborFramework for evaluating and improving agentsPythonApache-2.0 · Permissive6,840

20evcxr / evcxrInteractive Rust evaluation tools, including a Jupyter kernel, a Rust REPL and related shared libraries.RustLicense not declared6,529