The AI drafts looked finished. None of them were real.
A drafting step in our own back office was quietly returning templates instead of real drafts. No error, no alert. How we found it and what changed.
3 min read Provecta Group
On Monday, 29 June, we confirmed that every client document our new drafting step had produced up to that point was a fallback skeleton. Not a weak draft. A template, a few hundred characters long, with headings and placeholder lines, sitting in the system as if it were the real thing.
Nothing had errored. Nothing had alerted anyone. The step had run, a document existed, the status said drafted.
This post is about how that happens, because it will happen in your firm too if you automate anything with AI, and because the fix was only partly technical.
What we were building
Provecta's back office takes an engagement from proposal to delivery. One step drafts working documents for a client engagement: an audit write-up, a market segmentation, that kind of thing. The AI reads the engagement's materials and writes a first draft that a person then reviews before anything reaches the client.
When we tested it, the drafts came back short. Our first theory was that we were hitting the AI provider's rate limit by generating too many documents back to back. That theory was reasonable and wrong.
What was actually happening
A full draft took the model around 55 seconds to write. Our code gave each request 22 seconds before giving up. When it gave up, it did what we had designed it to do in an emergency: it fell back to a skeleton so the user saw something instead of an error.
The emergency path had become the only path. Every single draft had timed out and been replaced by the skeleton. The record we keep of each successful model call had no entries for this step at all, ever. That was the clue that settled it. If the model had written even one real draft, there would have been a row.
The fix was ordinary engineering. We trimmed the input, capped the output length and gave each request enough time to finish inside the limit our hosting allows. Drafts started completing in about 39 seconds, at several thousand characters each, written for the specific engagement. We also changed the skeleton's footer, which had been telling people the AI was not configured, so that it now says plainly when a draft timed out.
Why this is an accuracy problem and not just a bug
Most conversations about AI accuracy focus on hallucination: the model states something false. This was a different failure. The model never ran. The system stated, by its silence, that a draft existed. Anyone glancing at the list of documents would have believed it.
Developers already feel this. Stack Overflow's 2025 survey found that 46% of developers distrust the accuracy of AI tools. That distrust is usually aimed at the model. A lot of it belongs to the plumbing around the model: timeouts, fallbacks, retries and defaults that turn a failure into something that looks like success.
Where memory comes in
After the fix, the more important step was writing down what we learned, in the place every future AI session at Provecta reads before it works. The note is short. A real draft and a fallback can be told apart by length and by whether a successful model call was logged. Long generations need a time budget that matches the hosting limit. A fallback must say it is a fallback.
That note means the next person, or the next AI session, building a step like this starts from that knowledge instead of rediscovering it on a client's engagement.
To be honest about it, the memory did not catch this. A test did, and a person reading the output did. What the memory does is stop us paying for the same lesson twice.
What to check in your own automations
If your firm uses AI anywhere in a workflow, whether that is drafting quotes, summarising calls or writing follow-up emails, ask whoever built it three questions:
- What happens when the AI call fails or times out? Does the user see an error, or does something get filled in?
- If something gets filled in, does it say so on the output itself?
- Is there any record that shows how many outputs came from the AI and how many came from the fallback?
If nobody can answer the first question, assume the answer is "something gets filled in". Then go and look at the last twenty outputs, and pay attention to the short ones.
Run a quick health check on your operating setup at pgco.world/tools/os-health.