The proof-of-concept handbook: designing a 4-week PoC that actually answers something
Proof-of-concept work goes wrong in predictable ways. A working demo that answers no question. Five questions that all get answered badly. A success on the vendor's test data that fails on yours. A practical handbook for designing a tight four-week PoC that lands one clear answer.
By Mohammad Saqib Shah, Co-founder
A four-week proof of concept is one of the most useful tools available to a leadership team weighing an AI investment. It is also one of the most commonly misused. PoCs that fail to inform a decision do not fail dramatically; they fail quietly, by producing artefacts that look like progress without resolving the question that motivated them in the first place.
The failure modes are recognisable enough to design around. Here is what tends to go wrong, and what a PoC that earns its four weeks looks like instead.
The four common failure modes
First, the PoC ships a working demo but answers no question. The team has built something, it runs, the demo is well received, but nobody can say what the team now knows that they did not know before. Second, the PoC tries to answer five questions and answers none of them well. Resources spread thin across multiple hypotheses, each one explored just deep enough to be unconvincing. Third, the PoC succeeds on the vendor's or the team's curated test data and falls apart on the buyer's real data. Fourth, there is no agreed definition of done, so the PoC quietly extends from four weeks to eight to twelve, gathering scope as it goes.
All four are avoidable. None of them are avoided by accident.
Pick exactly one question, write it down
The first move is to commit to one question, in writing, before any code is written. The question should be specific enough that the answer is binary or close to it. Not “can AI help us with customer support” but “can a retrieval system answer 70% of tier-one customer questions, end-to-end, on tickets pulled from last quarter, with a deflection rate the support manager is willing to sign off on.”
Get the question signed off by the person who will act on the answer. The wording matters. If it cannot fit in a sentence, it is too broad, and the PoC will fail to land it.
Define done in advance
Before any work begins, write down what specifically would count as a yes, what would count as a no, and what would count as an iterate-and-retry. This is the single highest-leverage move available. A PoC without a pre-agreed definition of done becomes a perpetual project, because the team keeps finding the next thing that might tip the answer.
Define done as concrete thresholds wherever possible. Specific accuracy bands on specific data slices. A latency ceiling. A cost-per-call ceiling. A pre-defined sample of edge cases the system must handle. If the thresholds are arbitrary, that is fine, as long as they are written down and agreed.
Use real data from day one
Synthetic data, curated test sets, and vendor-provided examples are useful for hour one. They are dangerous from day two onward. The shape of real production data, with its missing fields, its inconsistencies, its accumulated idiosyncrasies, is the entire point of a PoC. A four-week PoC that does not touch real data until week three has spent two of its weeks learning nothing.
Get the data access conversation started before the PoC begins, not after. Data access is almost always the longest pole in the tent, and it is rarely an engineering problem. It is a procurement and security one, and it takes the time it takes.
Measure something the business cares about
Accuracy and F1 are convenient metrics for the engineering team. They are rarely the metrics the business actually cares about. A PoC measured only on technical metrics will produce a result that the technical team understands and the business does not. A PoC measured on the business outcome (time saved per ticket, deflection rate, cost per resolved query, hours of analyst work avoided) produces a result the leadership team can act on.
Pick the business metric first. Pick the technical metrics that map onto it second. Report both, but lead with the business one in the writeup.
Be willing to stop early
If the answer arrives in week two, stop. Write up what you found, deliver it, and return the remaining time and budget. The temptation to keep going (because the budget is allocated, because the team is engaged, because stopping feels like quitting) is significant and worth resisting. A PoC that closes early on a clear answer is a successful PoC, not a short one.
Stop is one of three valid outcomes
Every four-week PoC has three valid outcomes: productionise, iterate, or stop. All three are equally legitimate. The most common reason a four-week PoC turns into a twelve-week PoC is institutional discomfort with shipping a stop verdict. The team has done the work, the leadership team has committed the budget, nobody wants to write down “we tried it, the answer is no.”
Refusing to ship the stop verdict does not make the answer better. It just delays it, expensively. A PoC framework that builds in the stop outcome from the start (and treats it as a delivered result, not a failure) is one of the few practical ways to keep proof-of-concept work honest.