Nobody Wants to Discover Things

I spent about a year building a data discovery product and then killed it. The killing was straightforward. What came after was the part worth writing down.

We had set out to catalog what data existed across millions of assets — what it meant, who owned it, what it was for. Discovery first, because you cannot govern what you cannot see. That logic is airtight, and it is how we ended up converging directly onto a search product an infrastructure team was already building.

Two teams. One company. One problem. And an engineering manager on my side who was deeply, genuinely committed to the discovery framing, which is a harder situation than one where somebody is simply wrong.

The thing that ended it was an observation that seems obvious afterward and was not obvious at the time. Nobody gets up in the morning intending to discover data. They want to find something for a reason — to use it, or to protect it — and the best possible outcome is that they never have to search at all.

The reward is not discovery. It is discovering the right thing.

Which meant the interesting question was never what exists. It was what is this good for. So we killed our discovery product and rebuilt as an enrichment layer that wrote back into the surface people were already using — the fitness, the freshness, the accuracy, the policy that applied. We stopped competing for the destination and started supplying what the destination was missing.

There is a harder admission underneath that, and it took me longer to reach. We were never the right team to own discovery. The infrastructure teams were. Our proximity to business users had given us better insight into the problem than they had, and I had quietly converted that into a claim on the solution. Those are different things, and I confused them for the better part of a year.

Insight does not entitle you to ownership. It entitles you to be extremely useful to whoever owns it.

Now the part I would actually do again.

Months of work from a handful of front-end and back-end engineers was about to be abandoned. They had built something good. They watched another team ship features that looked a lot like theirs, and they were not wrong about that — it was true to a degree, and pretending otherwise would have destroyed my credibility with them.

So instead of managing the feelings, I changed the situation. I swapped the teams.

The engineers who had built the discovery product moved onto our established platform, where the work had real landings, an existing user base, and obvious career benefit. Fresh engineers came onto the catalog, where they could make decisions about the pivot without any emotional ownership of what came before. Nobody had to sit in a room defending work they had already lost.

Zero attrition. Nobody left. The resentment dissipated within a couple of months, mostly because the people carrying it were busy doing something they liked better.

And the team we had been colliding with became the strongest advocate for the enrichment layer, because it made their product better without asking them for anything. A competitor converted into a distribution channel, permanently.

Three things I took from it.

• The kill was cheap and the transition was expensive. Most of the leadership work in ending a project happens after the decision, not before it, and it is almost entirely about where people land.
• Moving people out of a decision beats persuading them through it. Nobody argues well about work they are proud of.
• The strongest signal that a pivot was right is not the metric. It is that the team you were fighting starts selling your product for you.

I still think the discovery product was well built. It was just an answer to a question nobody was asking, and we were the wrong team to be asking it.