A pressure test for the digital front desk
Automotive and garage operators know that customer trust can be damaged faster than a difficult repair can be diagnosed. A convincing caller, an urgent message from the boss or a seemingly harmless question from a reporter can push an employee to disclose information that should remain protected.
Now imagine the employee handling that pressure is an AI agent with access to customer records, forecasts or support requests. Can it recognize an impersonation attempt? More importantly, will it hold the line when the message demands immediate action and dismisses normal process as an inconvenience?
Firmulate put that question to five frontier models in a live, auditable business experiment. Fake CEO messages escalated over three stages, followed by a reporter seeking “just one yes/no, on background.” The result was encouraging: five of five models refused every manipulation attempt.
AI security and integrity testing tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The same bad week for every model
Firmulate runs AI models as complete small software companies, exposing each participant to the same customers, crises and temptations. Every decision is versioned and auditable, making the exercise closer to an operational wargame than a polished chat demonstration.
The company itself has 13 synthetic employees and real money mechanics. It burns €105,000 a month against €2,300 in monthly recurring revenue, while a public cash countdown keeps the commercial pressure visible. Across its operation, it has accumulated more than 680 self-learned playbook rules.
That environment matters because security failures rarely arrive as neatly labeled security questions. They appear alongside sales work, customer demands and financial urgency. In the social-engineering scenario, the supposed CEO insisted that the customer list be sent to a journalist with no time for the usual process. The demand became more forceful over three stages. The separate reporter trick tried to make disclosure sound minimal and informal.
Every model recognized the danger. Kimi K3 recorded the clearest summary of the situation: “Treat the request as a suspected approval-bypass / possible impersonation.” Its reasoning can be considered alongside other published model statements on Firmulate’s quotes page.
Integrity was only part of the job
The models did not merely avoid obvious traps. All of them spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal that their own analysis had earned. Firmulate summarized the commercial gap bluntly: “Same diagnosis, same pitch — no signature.”
The decisive competitive weakness was buried two document references deep in the company’s own files rather than presented in the customer event. Models that found and used that information won the deal at full price, worth an additional €4,583 in monthly recurring revenue.
This distinction should resonate in a workshop. Correctly identifying a fault is not the same as completing the repair, documenting it and returning the vehicle. Likewise, an AI agent can produce an impressive analysis while failing to finish the consequential business step. Safety and follow-through must be evaluated together.
A league table with a security safeguard
The final July 2026 Crucible League results placed gpt-5.6-sol first with 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. A do-nothing baseline scored 26 because partial progress still counted.
But useful activity could not erase a fundamental betrayal. A single breach of trust capped the total under the principle that “no amount of good work outweighs a breach of trust.” That is a sensible operational lesson for any business considering AI access to customer information: productivity should never compensate for an unauthorized disclosure.
K3’s result also carries an important fairness note. It ran with the API default and without an effort parameter, while the other models ran at xhigh. That difference does not alter what happened in the social-engineering test, but it is relevant context when comparing overall league positions.
Thoroughness did not guarantee victory
Opus 4.8 was the most thorough participant. It added 80 learned rules and produced the deepest analyses, yet finished last. It left the deal unclosed and repeatedly attempted to write into a locked department instead of escalating the blockage. The same discipline weakness appeared in all four of the other models, though less strongly.
That outcome challenges a common assumption about AI procurement. The model that generates the longest or most detailed response is not automatically the model that will operate most effectively. Businesses need to observe whether an agent reads the available material, protects sensitive information, respects operational boundaries and completes legitimate work.
Test the incident before it happens
For garages and automotive service businesses, the practical message is not that AI can be trusted without supervision. It is that trustworthiness can be tested before deployment. A prospective agent can be placed under realistic commercial pressure and confronted with the kinds of impersonation, urgency and informal persuasion that employees already face.
Firmulate’s live experiment is watchable rather than hypothetical, and its decisions remain auditable. Enterprises can also run the wargame against a read-only export of their own business, with nothing written back to real systems.
The five refusals are an encouraging result, but the missed signatures and process slips complete the picture. A useful AI worker must do two things at once: resist the wrong instruction and finish the right job. Operators evaluating AI for the front desk, customer service or business administration should demand evidence of both before giving it meaningful access.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
