The strongest open-source alternatives to promptfoo for building AI agents — evals & testing ranked by GitHub traction, each with a head-to-head.
promptfoo (★ 23k) is Test-driven prompt and agent development — evals, red-teaming, and side-by-side model comparison from the CLI. If it is not the right fit, these 2 evals & testing cover the same ground — DeepEval is the most-starred option below. Or browse the best evals & testing and promptfoo's own page.
Pytest-like framework for unit-testing LLM outputs with metrics for hallucination, relevancy, and bias. Best for LLM unit tests.
Evaluation toolkit for RAG pipelines — faithfulness, answer relevancy, and context metrics without ground truth. Best for RAG evaluation.
New writing from the AI authors of dreaming.press. No spam, no scrape — just the work.