Build Your Eval Suite Before You Build Your Feature

The feature ships. The model is great. And the first time it does something wrong in front of a customer, nobody can tell you whether it's getting better or worse. That's what happens when the eval suite comes after the feature. Usually it never comes at all.

Here's the pattern I keep seeing. Teams build an AI feature because the model demos well. The demo is the entire quality bar. The feature goes to production, and every decision about it afterwards is made on vibes — does it feel better, did the vendor say the new model is better, did a customer complain? The team is flying a plane with no instruments, and the plane is the model, which is the one component that changes without asking.

The fix is backwards from intuition: build the eval suite before you build the feature. The eval is the foundation, not the final step.

What an eval suite actually is

An eval suite is a test set for AI. A collection of inputs with known-good outputs, run against the model, scored for how often the model gets it right. It's the same shape as a test suite for code — a set of cases that must pass — with one difference: the outputs are fuzzy. The answer can be right in several ways, so the scoring has to be judgment-based, which means the eval suite is part test, part rubric.

The suite has three parts. The golden set: a few hundred cases, hand-curated, covering the important scenarios and the edge cases, each with a known-good answer. The regression set: cases that caught real bugs in the past, kept forever, so a fix never silently re-breaks. And the rubrics: the definitions of what “good” means for each task type, written down, so scoring isn't a mood.

That's the whole suite. It's not exotic. It's the AI version of what software teams have done for forty years, and skipping it is the same decision as shipping code with no tests — it works until it doesn't, and the cost shows up later.

The eval suite is your test suite. The model is your code. And like code, the model changes without telling you — the suite is how you find out.

Why it has to come first

The eval comes first because it changes every decision that follows.

Build the suite before the feature and the feature gets built against a target. You can try two models and pick the one that scores better on your cases instead of the one that demos better. You can tune the prompts against the rubric instead of against a feeling. You can say no to a “great” model that fails your edge cases. The eval turns model selection from a demo contest into a measurement.

Build the eval after the feature and you get the retroactive version of all of that. The model is already chosen, the prompts are already written, the feature is already in front of users. The eval that arrives late can only document the mistakes it was built to catch. It can tell you the model is bad. It can't tell you which one to have picked.

The other reason for first: the eval is how the team learns what “good” means. Writing the golden set forces the conversation — what are the edge cases, what does a wrong answer look like, what's the cost of each kind of error. That conversation is the actual specification of the feature. Teams that write the eval first discover the requirements before they build. Teams that skip it discover the requirements after the incident.

The minimum viable suite

The objection is always the same: building an eval suite sounds like a project, and the feature has to ship. The minimum viable suite answers that. It's fifty cases and a weekend.

Fifty cases, curated in an afternoon with the domain owner: the ten most common requests, the ten hardest, the ten most dangerous, the ten that look similar but need different answers, and ten that broke similar systems before. One weekend to run them, score them, and write down the rubric. That's the seed. The suite grows from there — every production bug becomes a regression case, every new task type gets its own golden set, every model upgrade gets scored against the whole thing.

The fifty-case suite won't catch everything. It will catch the big stuff, it will force the spec conversation, and it will give you a number to watch. The team with fifty cases and a rubric is infinitely better off than the team with a great demo and nothing else.

The rule that makes it stick

The eval suite dies the same way every test suite dies: when it stops being used. The rule that keeps it alive is simple — no model change ships without an eval run. New model version, new prompt, new temperature, new anything: run the suite, compare the scores, and only ship if it's better or equal. That's the whole discipline, and it's the same discipline that keeps tests honest in software.

The feature you're building right now will be maintained by the eval suite you build this week. The model will change, the vendor will update, the data will drift, and every one of those moments is a decision that the suite makes answerable. Build it first, keep it small, and make it the gate. The demo told you the model could do the task. The eval tells you whether it still can, next month, after everything changed.

Build the suite before the feature. Fifty cases, one weekend, and a rule that nothing ships without a score. That's the whole practice, and it's the difference between AI you trust and AI you hope for.

Think this argument fits your event? Tell me about the room — the calendar is selective.