From forty tools to one layer

Forty licences. Nine pilots. Each one works. The company has not got more intelligent.
This is the most common state I walk into now, and it is worse than having nothing, because the money is spent and the organisation believes it has an AI capability.
Why forty working tools do not add up
Each tool carries its own copy of the context. Its own idea of what a customer is, its own slice of the documents, its own connection to one system, its own notion of what good output looks like.
Nothing one tool learns is available to any other. The work of connecting a model to the company’s actual knowledge gets done forty times, badly, by forty teams who each considered it the boring part of their project.
Then the vendor changes something, and you find out that each of the forty integrations was load-bearing in a way nobody wrote down.
The engineering literature saw this coming
Ten years ago a team at Google wrote the paper that describes what you are now living in. Its opening observation is the one to carry into your next architecture meeting: developing and deploying machine learning systems is relatively fast and cheap, but maintaining them over time is difficult and expensive.
They catalogue the specific ways it goes wrong: boundary erosion between components, entanglement where changing anything changes everything, hidden feedback loops, undeclared consumers who depend on your output without telling you, data dependencies that nobody tracks, configuration sprawl, and the accumulation of glue code around systems treated as black boxes.
That paper is about machine learning systems at Google in 2015. Every failure mode in it applies, unchanged, to a company that has bought forty generative AI tools in eighteen months, except that the glue code is now being written by people who do not think of themselves as building a system at all.
The phrase I take from it is hidden debt compounds silently. Nothing in your dashboard will tell you this is happening. The tools all still work.
What actually has to be shared
Four things, and the list is short on purpose.
The data and retrieval layer. One answer to where the company’s knowledge lives and how a system gets at the right piece of it. This is the one that pays for itself fastest and the one that is hardest to fund, because it belongs to no single use case.
Identity and permissions. What a given user is allowed to see has to be enforced in one place. Resolved per tool, you will eventually have a system that cheerfully summarises a document the person asking should never have seen, and you will find out from the person.
Evaluation. How you know whether a change made things better. Without shared evaluation each team ships on vibes, and over time the honest ones look worse than the optimistic ones.
Observability and cost. What is being spent, by whom, on what, and how often the thing is wrong.
What does not have to be shared
The model. Use several, change them, they are a commodity and the switching cost should be low by design.
The interface. A tool for claims assessors should look like the claims system. One shared chat window into which everything is funnelled is a platform team’s idea of a good outcome and nobody else’s.
The use cases themselves. This is where most platform efforts overreach. The goal is a shared floor, not a shared building. A central team that wants to own every use case becomes the bottleneck it was created to remove, and I have watched that happen twice.
The order to build it in
Nobody gets to build all four at once, so the sequence matters more than the list.
Start with observability, even though it is the least interesting. You cannot argue for any of the rest without knowing what the forty tools currently cost and how often they are wrong, and that argument is the one that unlocks the budget for everything after it. It is also the only one you can build without asking any other team to change anything.
Then retrieval, because it is where the duplicated effort is largest and the quality difference is most visible to users. A team that switches to shared retrieval should see their own tool get better the same week, which is the only argument for adoption that reliably works.
Permissions next, and sooner if you are in a regulated sector, where it moves to first.
Evaluation last, which is contentious, and my reasoning is practical rather than principled. Evaluation is the hardest to design before you know what people are actually building, and a shared evaluation framework imposed early tends to measure what was easy to measure in month one.
The layer I built that nobody used
I have been on the wrong side of this argument too.
A client where I pushed hard for the shared layer, and we built it properly. Good retrieval, proper permissions, clean evaluation. It was the right architecture and I would build the same thing again.
Two teams used it. The rest carried on with their own tools, because using the layer required a two-week onboarding, a review by the platform team, and a quarterly capacity conversation, while the vendor alternative took an afternoon and a credit card.
The layer lost on friction, not on merit. We had built a better floor and put a door in front of it.
What I would do differently is budget as much for the on-ramp as for the thing itself, and treat adoption by other teams as the actual deliverable rather than a thing that happens afterwards. A shared layer with two users is a tool with extra governance, and an expensive one.
How to tell where you are
Pick a question that spans two of your tools. Something a real employee would ask that requires knowledge sitting behind both.
If no tool can answer it, you have forty tools. If one can, ask how it got the other half of the context, and if the answer is that somebody copied the data across in March, you have forty tools and a cascade forming.
Then ask who would know if one of them started being wrong. If nobody has an answer, that is your first piece of work, and it is observability rather than anything glamorous.
The capability layer is chapter seven of The AI Advantage. This follows When to stop an AI initiative. Next, three words that get used interchangeably and should not be: Copilot, workflow, agent.