We Researched Every LLM Eval Tool So You Don't Have To

We changed a prompt in our LLM scoring system and scores shifted. The feedback tone felt different. But "felt" is a dangerous word in production. Did it get better or worse? Along which dimensions? How many cases were affected? We had no idea — and figuring out how to stop being blind took us down a rabbit hole of tool comparisons, community debates, and hard-won opinions.

Why Did We Go Looking for LLM Eval Tools?

We're building an LLM scoring system — the kind where a language model reads exam responses, writes feedback, and assigns grades.

The system works. The model reads inputs, generates commentary, returns scores. Looks legit.

Then we tweaked a prompt.

After the change, scores shifted. The tone of the feedback felt different. But "felt" is a dangerous word in production. Did it get better or worse? Along which dimensions? How many cases were affected?

We were in an uncomfortable spot: the system was live, and we had zero idea whether its quality was improving or degrading.

This isn't a research problem. It's an engineering problem. You wrote the prompt. Now how do you know it's any good?

What Are the Seven Dimensions of an LLM Product?

We went down the rabbit hole and mapped out seven dimensions every LLM product needs to care about:

Model selection — Which LLM to use
Prompt engineering — How to write, manage, and iterate prompts
Output quality — Are the results correct and good?
Cost — How much API spend is this burning?
Latency — How long are users waiting?
Reliability — Is the system dependable?
Safety — Hallucinations, bias, data leakage

Each dimension has an MVP tier and a "serious" tier. For quality, the MVP is 20 golden examples you manually check. The serious tier is LLM-as-Judge + automated eval pipelines + CI gates.

Our take: start with the minimum viable Quality + Safety setup, expand from there. Most teams ship with nothing — being able to see your problems puts you ahead.

For us, dimension three mattered most. When you're building a scoring system, quality is the product.

How Do LangFuse, Braintrust, and Datadog Compare for Eval?

Once we decided to build an eval practice, the next question was: what tools? This turned out to be more painful than expected.

LangFuse: The Default Choice with Mediocre Eval

LangFuse is the default answer in LLM observability right now. 15k+ GitHub stars. Acquired by ClickHouse for $400M in early 2026. Khan Academy and Canva use it.

The tracing is genuinely good. Every LLM call — input, output, token count, latency, cost — all captured, filterable by user, session, prompt version. Prompt management works too: version your prompts in the UI, the SDK caches them, and you can change prompts without redeploying code.

The eval experience, honestly, is mediocre. You write code to run experiments. It's not the "click and compare on one page" experience some platforms offer.

Braintrust: Better Eval, But It's a Lab Not a Factory

Braintrust just raised an $80M Series B and positions itself as "evaluation-first."

The eval UX is better. One-page experiment comparison, a solid Playground, and a genuinely generous free tier — 1M spans/month, unlimited users. But production monitoring isn't LangFuse's level. It's more of a lab than a factory.

Datadog: Great Generalist, Doesn't Understand LLMs

We considered Datadog. Great general-purpose tracing. But it doesn't understand LLMs — no token cost calculation, no prompt management, no eval. It's the difference between a general practitioner and a specialist.

What Does the Community Say About These Tools?

Reddit discussions were illuminating. On r/LangChain, LangFuse was called the "most serious self hosted open source platform." Someone on r/AI_Agents nailed the core tension:

"Observability tells you what happened, not how to fix it."

That's the sentence that stuck with us. Another engineer shared their practical approach — instead of relying on dashboard alerts, they pin 8 canary scenarios and rerun them after every prompt change. Their quote:

"Caught more drift this way than langfuse alerts ever did."

Data export is a real pain point. As one redditor put it:

"Getting the data OUT to do cross-session analysis is a pain."

Hard to argue with that one.

Side-by-Side Comparison

Dimension	LangFuse	Braintrust	Datadog
Tracing	★★★★★	★★★★	★★★★
Eval	★★★	★★★★★	★
Prompt Management	★★★★	★★★★	✗
Open Source	✓ (MIT)	✗	✗
Self-hosted	✓	✗	✗
Pricing	Free (self-hosted)	Generous free tier	Expensive
LLM-specific	✓	✓	✗

Why Did We Pick LangFuse Over the Others?

We went with LangFuse Self-hosted.

The reasoning:

One platform covers 80% of our needs — Tracing, Prompt Management, and Eval are all there.
MIT license, self-hosted for free, data stays in our VPC.
Good data portability — CSV/JSON/API export, official migration cookbook from Cloud to Self-hosted.
We don't want three platforms. One that's good enough beats three that are perfect individually.

Braintrust has a better eval experience, but adding an entire platform for one capability isn't worth the overhead. Datadog knows infrastructure, not LLMs — wrong tool for this stage.

What Four Opinions Did We Form After the Research?

1. Observability tells you what happened, not how to fix it

To borrow the Reddit quote again. Tracing is infrastructure, not a destination. Most teams plug in tracing, stare at the dashboard, feel informed, and do nothing. Data isn't insight. Analysis is.

2. Most teams ship with nothing

This isn't a joke — it's the reality. Many AI products run without eval, without tracing, without golden sets. No judgment if your use case tolerates it. But if you're building something where quality matters — a scoring system, a medical assistant, a legal summarizer — you can't operate blind.

3. One platform is enough. Don't collect three.

LangFuse for tracing + prompt management, Braintrust for eval, Datadog for infrastructure — sounds elegant, until you maintain it. Our advice: pick the one that covers the most ground and accept its weaknesses.

4. There's no silver bullet. The space is still early.

LLM eval moves fast. The tools of 2025 may be irrelevant by 2027. LangFuse itself was just acquired by ClickHouse — who knows what the roadmap looks like in two years. Don't over-invest in any single platform. Keep your data portable.

How Do You Build an LLM Eval Pipeline from Scratch?

If you're building an LLM product and want to go from "it runs" to "it's reliable," here's the simplest path:

What's the bare minimum?

10 golden examples — hand-labeled correct answers, covering typical and edge cases.
Manual check — after changing a prompt, run the golden set and eyeball the results.
That's it. No platform, no code, done in an afternoon.

What Are the Three Stages?

Stage 1: Tracing (1–2 days) Wire up LangFuse. Log every LLM call's input/output/cost/latency. For the first time, you'll see what your system is actually doing.

Stage 2: Prompt Management (2–3 days) Move prompts out of your codebase and into LangFuse. Change a prompt without redeploying. Takes effect in seconds.

Stage 3: Eval Dataset (3–5 days) Build your golden set into LangFuse, configure an LLM-as-Judge. Automatically benchmark before every prompt change. Prevent regressions.

Three stages, roughly a week. After that, it's continuous iteration: Observability → Evaluation → Prompt Management → repeat.

What Did We Learn from This Journey?

From Prompt Change to Production Confidence

Eval isn't a one-time project. It's ongoing engineering discipline. We're still somewhere in Stage 1, but at least now we know which direction to walk.

If you're on a similar path, hope this saves you some time. No profound insights here — just the thinking process of an engineering team facing a real problem.

FAQ

What is LLM Eval and why does my AI product need it?

LLM Eval is the systematic practice of measuring the quality of large language model outputs. Without it, you can't tell whether a prompt or model change improved or degraded results — which is unacceptable for quality-sensitive applications like scoring, medical, or legal systems.

LangFuse vs Braintrust: which is better for LLM evaluation?

LangFuse covers more ground (tracing + prompt management + eval) and is self-hostable; Braintrust has a superior eval UX but lacks production monitoring and self-hosting. For most teams, the broader platform is the pragmatic choice.

What's the minimum I need to start evaluating LLM outputs?

10 hand-labeled golden examples + manual review after every prompt change. No platform required, done in an afternoon. This is the real MVP — automate only after this habit is established.

What is a Golden Set and how do I build one?

A Golden Set is a curated collection of input-output pairs with human-annotated expected results, covering typical and edge cases. Pick 10-20 representative examples from production data, manually label the expected outputs, and use them to validate changes.

What is LLM-as-Judge?

LLM-as-Judge is a technique where one LLM evaluates the outputs of another. You define scoring dimensions (accuracy, relevance, tone) and the judge model scores outputs automatically — replacing manual spot-checks with scalable, consistent evaluation.

This post is based on our own research process. Here are the key sources: