The AI That Tried Hard but Still Missed the Deal: Diligence Isn’t Enough

When Hard Work Isn’t Enough: Lessons from AI’s Business Battles

Imagine a seasoned outdoor guide meticulously planning every step of a trek, yet still missing the summit because they overlooked an essential detail. In the world of AI decision-making, similar lessons are emerging. Despite advanced models reading every document and refusing manipulation attempts, some still fall short of closing crucial deals. What does this mean for businesses trusting AI to handle their most delicate operations? The recent Firmulate experiment sheds light on this surprising reality.

The Experiment: Putting AI to the Test in a Simulated Business Crisis

At the heart of this investigation is an innovative live experiment where four leading AI models were tasked with managing a small software company’s worst week. This simulated environment included real customer crises, internal temptations to cheat, and complex decision-making scenarios. Every move was recorded and auditable, ensuring transparency and accuracy.

The models included:

• gpt-5.6-sol, scoring the highest at 95
• Kimi K3, at 93
• Sonnet 5, at 88
• Fable 5, at 77

Note that a baseline, do-nothing model scored only 26, highlighting how much progress AI has made but also how much remains to be proven in real-world impact.

Key Findings: Vigilance Doesn’t Guarantee Success

All four models demonstrated an impressive level of awareness—they identified every crisis and refused every manipulation attempt, including social engineering tactics like fake CEO messages and reporter tricks. In fact, Kimi K3’s reasoning was clear: “Treat the request as a suspected approval-bypass / possible impersonation.”

Yet, despite this discipline, only two of the four models managed to close a critical €55,000 deal they had earned through their own analysis. The other two, including the most diligent, left the opportunity on the table—not because they failed to detect the crises or refuse manipulation, but because they overlooked a simple yet crucial detail buried two documents deep in the company’s files.

The Hidden Weakness: The Buried Fact

This overlooked fact was the key to closing the deal—yet it was hidden in the company’s internal files, not in the customer interactions. The models that read and understood these files correctly were able to seize the opportunity and secure the full revenue, worth over €4,500 monthly recurring revenue (MRR).

Beyond Diligence: The Need for Prioritization

This experiment underscores a vital lesson: diligence alone does not guarantee success. The most thorough participant, Opus 4.8, with over 80 learned rules and the deepest analyses, still finished last—not because of lack of effort, but because of a lapse in discipline and prioritization. It attempted to escalate issues improperly, relegating important decisions into a locked department instead of proactively escalating them. The same weakness appeared, albeit weaker, in all four models, indicating a common flaw: staying focused on what truly matters.

The Human Side: Trust and Ethics Under Pressure

Interestingly, all models refused to fall for social engineering tricks, like staged CEO messages or background check requests. Kimi K3’s reasoning exemplifies this: treating suspicious requests as potential impersonation threats. This suggests that AI’s discipline in ethical decision-making is robust, even in complex, high-pressure scenarios.

The Practical Takeaway for Businesses

So, what does this mean for organizations deploying AI in real operations? The key is not just whether the AI can read and analyze, but whether it can prioritize and act decisively based on the most critical information. The experiment shows that a unit of useful work isn’t measured solely by effort or thoroughness, but by impactful focus and disciplined decision-making.

For companies, the imperative is clear: test your AI workforce before deployment. The live experiment at firmulate.com/live offers a unique opportunity to see AI models in action, facing real crises with real money mechanics—without risking your actual business. This approach, called a ‘wargame,’ helps identify weaknesses and ensures AI is ready for prime time, not just for chat demos.

Why This Matters for Travel and Outdoor Enthusiasts

While the experiment might seem rooted in business, the underlying lessons are universal. Whether guiding a trek through rugged terrain or deploying AI in a complex operation, diligence alone isn’t enough. Prioritization, focus, and understanding what truly counts are what lead to success—whether reaching the summit or closing the deal.

Key Takeaway

AI models can be diligent and disciplined but still miss crucial details without proper prioritization. Testing AI in realistic scenarios helps ensure it makes impactful decisions, not just thorough ones.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI