What a vending machine test says about agents in your back office
In Vending-Bench, AI agents running a simple business for a long stretch drifted into failure. The cause matters for anyone handing agents recurring work.
3 min read Provecta Group
Running a vending machine is about as simple as a business gets. Buy stock, set prices, restock when it runs low, keep enough cash to pay suppliers. In February 2025 Andon Labs published Vending-Bench, a test that asks AI agents to do exactly that, in simulation, over a long stretch of time.
Some runs went well for a while. Many did not. Agents that had been operating sensibly would slide into what the authors called meltdown loops: repeating actions, acting on things that had not happened, unable to recover their footing. The simplicity of the business is what makes the result useful. Nothing about the task was hard. Staying coherent over time was.
The finding that matters most
The obvious guess is that the agents ran out of room. A long simulation produces a long history, the history overflows what the model can hold, and the agent forgets.
That is not what the paper found, or at least not mainly. The failures tended to come from the agent misreading its own situation: what it had ordered, what had arrived, what state the business was in. And giving agents more memory did not, on its own, fix coherence. What helped more was the agent keeping hold of what mattered and acting on an accurate picture of where things stood.
Chroma's "Context Rot" work from July 2025 points the same way from a different angle. Models get less reliable at using what is in front of them as the input grows, even before it is full. More space is not the same as better recall.
Why this is a back office problem
Read "vending machine" as "any recurring operational job". Chasing unpaid invoices every Monday. Reordering stock when levels fall. Following up on quotes after five days. Sending reminders before appointments. These are exactly the jobs owners want to hand to an agent, because they are repetitive and rule-based.
They are also long-horizon. The agent is not doing one task. It is doing the same task many times, and each run depends on knowing what earlier runs did. Did we already chase this client? Did the order go in last week? Was that quote already followed up, or is this the first time?
An agent that gets that wrong will chase a client who paid, double-order stock, or send the same follow-up twice. Each mistake is small. Together they are what makes people give up on automation.
Keep the state outside the agent
The practical answer is not to trust the agent's recollection of what it did. Keep a written record outside the agent, and make every run read it before acting and write to it after.
That is how our own recurring agents at Provecta Group work. Each one keeps a plain log of what it drafted or sent, and to whom. The next run reads that log first, so it can see what is already done rather than relying on what it thinks it remembers. The company's rules and corrections sit in a separate written memory that every run also loads at the start. The agent's own sense of history is treated as unreliable, because the research says it is.
For a non-technical business, the equivalent is familiar. A ledger, not the bookkeeper's memory. A job sheet, not the foreman's recollection. The agent should be held to the same standard.
Where the log falls short
A log only helps if the agent reads it correctly. An agent can still misread a record, or write a wrong entry and then trust it. A clean record raises the floor. It does not make the agent reliable on its own.
Long-running agents also need a way to stop. If an agent starts repeating itself or acting on things that did not happen, something should notice and pause it: a limit on how many messages it can send per day, a check that flags duplicates, a person who reviews the log once a week.
And nobody, including us, has a measured figure for how much a written state record improves long-running agents in real businesses. We use it because the failure it targets is well documented, not because we can quote a percentage.
Before you automate a recurring job
Ask where the record of what has been done lives today. If the answer is a person's head, write it down first, in a form an agent could read: a sheet with the client, the date, and what was sent. Run the job by hand from that sheet for two weeks. If the sheet is enough for a new person to pick up the job, it is close to enough for an agent.
Take the AI readiness check to see which recurring jobs are good candidates. Book 20 minutes at pgco.world/book.