Skip to content
Menu
← All articles

9 November 2026 · 4 min read

Why AI prototypes stall before production

A prototype that impresses a room is the easy part. Many companies reach that point within weeks, then spend months unable to get from the demo to something people rely on every day. The reasons are rarely about the model. They are about everything around it. Here are the ones we see most often, and what we would do about each.

Integration with real systems and data

A demo usually runs on a handful of clean examples that someone picked by hand. Real work lives in a CRM, an ERP, a mailbox, a shared drive, and the data there is incomplete, duplicated and inconsistent. The system has to read from these sources and often write back to them.

Plan the integration first, not last. List which systems the process touches, what each one exposes, and who can change it. Then build the prototype on the real connections, even if it covers only a small slice of the work.

Permissions and access

In a prototype, one person’s credentials are often enough. In production, the system must respect who may see what. A tool that summarises documents for everyone must not reveal contracts that only the finance team may read.

Decide early how the system identifies a user and which rules it enforces. It is much cheaper to design access rules at the start than to add them to a system that has already been shown to the whole company.

Edge cases

Demos cover the typical case. Daily work is full of the unusual one: a scanned document at an angle, a customer who writes in two languages, a request that is missing half its details. A system that handles ninety easy cases and fails silently on the tenth will lose people’s trust quickly.

Collect real examples, including the awkward ones, from the people who do the work. Decide in advance what the system does when it is unsure. Passing the case to a person, with a clear note about what was unclear, is almost always better than guessing.

Evaluation

“It seems to work” is not a test. Without a fixed set of examples and an agreed way of judging the output, nobody can say whether a change made things better or worse. Teams end up arguing from memory.

Build an evaluation set early, from real cases with known good answers, chosen with the person who owns the process. Run it after every change. It does not need to be large to be useful, and it becomes the shared definition of correct.

Monitoring

A system that worked on launch day can drift. Inputs change, a connected system is updated, a document format shifts. Without monitoring, the first sign of trouble is a complaint.

Record what the system received, what it produced and what a person did with it, taking care over personal data. Review a sample regularly, and set alerts for failures that are easy to detect. Decide who looks at this, and how often.

Cost

A prototype serving ten requests a day says nothing about the bill for ten thousand. Usage cost, infrastructure and the time of the people who maintain the system all grow with use.

Estimate the running cost with realistic volumes before committing. Choose the simplest approach that meets the quality bar, and measure the cost per task so it can be compared with the value of the work removed.

People and process

This is the one most often underestimated. A system that changes how people work needs their cooperation. If staff do not understand it, do not trust it, or were never asked, they will route around it.

Involve the people who will use it from the first week. Be clear about what changes in their day, who reviews the output, and who they ask when something looks wrong. Give one person ownership of the result on the client side, and one on ours.

What to do about it

Most of these problems are cheaper to handle at the start than at the end. Our approach is to put a thin but real version into the actual environment early, with real data, real permissions and a small group of real users. It will be rough. It will also show the problems that a demo hides, while they are still inexpensive to fix.

Then we widen it step by step, measuring against the evaluation set, keeping a person in the loop where the risk calls for it, and handing over a system that someone inside the company is prepared to own.

If you have a prototype that has stopped moving, or you want to avoid getting there, we would be glad to look at it with you. The first conversation is free and non-binding, and you can start on the contact page.

Have a problem AI might solve?

Describe it in a few lines. The first conversation is free and non-binding, and we usually reply within one to two business days.

Let's talk