
In the high-stakes world of AI management, how do you measure true competence? Surprisingly, even a so-called ‘do-nothing’ baseline scores 26 out of 100 on a leading test. This reveals key truths about AI accountability and performance—lessons that matter whether you’re managing a team or deploying AI systems in business.
Get sports gear delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
Understanding the Benchmark: More Than Just Scores
At the heart of this experiment is a straightforward question: what happens when AI models are tasked with running a small software company through its worst week—complete with crises, manipulations, and ethical dilemmas? The answer provides a transparent look at AI’s real capabilities, not just what they can generate in chat but how they handle complex, time-sensitive decisions.
The benchmark, conducted by Firmulate, pits four frontier AI models against a simulated company facing real-world challenges. Every decision made during this week is versioned, auditable, and designed to test not only problem-solving skills but also trustworthiness and discipline under pressure.
The Surprising Power of a Baseline
One of the most revealing findings is the performance of a ‘do-nothing’ baseline—simply running the model without any specialized effort or tricks. This baseline scored 26 points out of 100, illustrating that even minimal engagement counts for something. Partial progress is recognized, and crucially, any breach of trust—like signing a fake deal—caps the total score at that breach point, no matter how well other tasks are handled.
Products Worth Considering

Garmin Instinct 3 AMOLED Rugged GPS Smartwatch, 50 mm, Charcoal

Garmin Forerunner 265 Running Smartwatch 46 mm Black/Powder Gray

YETI Roadie 15 Hard Cooler
As an affiliate, we earn on qualifying purchases.
Why This Matters for Business and AI Deployment
Many people assume that the true test of an AI system is how well it generates human-like text. But in business contexts—support, sales, compliance—the real questions are: does it finish what it starts? Does it prioritize honest, reliable work over shortcuts? And critically, does it stay honest when under pressure?
Firmulate’s live experiment, viewable at firmulate.com/live, demonstrates that all models in the test identified every crisis and refused manipulative tactics designed to deceive them. For example, fake CEO messages escalating over multiple stages were refused by all five models, with one (Kimi K3) explicitly treating such requests as potential impersonation.
Deep-Read and Deal
The most decisive advantage went to models that read deeper into internal documents—two references down in the company’s files—rather than just reacting to surface-level cues. Those models secured the €55,000 deal at full price, a simulation of closing a key business contract, earning an extra €4,583 monthly recurring revenue (MRR).
The Human-Like Limitations and Trust Boundaries
Interestingly, even the most thorough model—Opus 4.8—fell short in closing the deal, leaving a chance on the table when discipline slipped into internal channels instead of escalating issues properly. All models demonstrated that they can spot crises and refuse manipulative tactics, but discipline and internal process adherence remain challenges.
Fairness and Testing Conditions
It’s worth noting that one model, K3, ran without an effort parameter (default API setting), while others used higher effort levels, hinting at how configuration impacts performance. This underscores that performance isn’t just about raw intelligence but about how models are set up and used.
From Lab to Business: Wargaming AI Before You Hire
For enterprises, the takeaway is clear: running AI models through this kind of ‘wargame’—a simulated week of real crises, manipulations, and strategic choices—provides a clearer picture of their true management capabilities. You can test AI systems against your own business scenarios without risking real damage, thanks to tools like the Firmulate Pilot.
In an era where AI is touching everything from CRM to support queues, it’s not enough that they write well. The focus should be on whether they complete tasks honestly, read critical information first, and stay disciplined when tempted to cut corners.
The Bottom Line: Trust and Performance in AI
Ultimately, the benchmark’s core message is that even a do-nothing baseline achieves a score of 26, setting a floor for honest, minimal effort. No amount of perfection in problem-solving can outweigh a breach of trust, which caps the performance at that level. True AI management isn’t about the highest chat scores but about integrity, reliability, and discipline—traits that this transparent, watchable experiment demonstrates well.

The real test of AI in business isn’t how well it chats but whether it can handle crises, read critical files, and stay honest under pressure. Firmulate’s benchmark reveals that even minimal effort registers meaningful progress, but trust remains the ultimate currency for AI performance.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
