You have a scope: one workflow, one owner, one number. The temptation now is to open an editor and start building the thing. The discipline that separates strong FDE teams is what they build first, and it is not the thing.

It is the harness that measures the thing.

#Evals first: the argument in one scenario

Imagine two FDEs given the same task: extract five fields from incoming insurance claims using an LLM.

The first builds the extractor immediately. It looks great on the four claims they tested by hand. Two weeks later an operator reports wrong policy numbers. They adjust the prompt. Something else quietly breaks. Every change is now a gamble, and confidence in the system decays with each surprise.

The second spends day one collecting fifty real claims and writing down the correct five fields for each, by hand, with the operator's help. That afternoon they wire a script: run the extractor on all fifty, compare against the answers, print a score. Only then do they write the extractor. Now every prompt change gets a number in thirty seconds. Improvements are provable. Regressions are caught before any operator sees them.

The second FDE is slower for one day and faster forever after. That trade has a name in this field: evals before demos.