Anyone who has sat in silence knows the feeling: you did everything right — the reading, the reflecting, the preparation — and yet the outcome didn’t come. The journal is full, the insights are deep, but the letter was never sent, the conversation never had, the forgiveness never asked for. In contemplative traditions this is an old teaching: effort is not the same as completion. Volume is not the same as impact.
Open a free Amazon Business account
Business pricing, bulk buying and tax-exempt orders.
As an affiliate, we earn on qualifying purchases.
It turns out machines can fail at this in exactly the way people do. And we can now watch it happen, in public, with a scoreboard.
The Same Worst Week, Four Different Minds
At Firmulate, an unusual experiment has been running: four frontier AI models were each given the same job — running the same small software company through its worst week. Same customers, same crises, same temptations to cut corners. Only the model changed. Every decision was versioned and auditable, so nothing could be hand-waved afterward.
The final league table from the July 2026 “Crucible” run tells a story that will feel familiar to anyone who has worked on themselves: gpt-5.6-sol finished first at 95, Kimi K3 followed at 93, Sonnet 5 scored 88, another Sonnet variant 77 — and Opus 4.8 came in last at 73. For perspective, doing nothing at all scores 26; and a single breach of trust caps the total, because, as the experiment’s own rule puts it, “no amount of good work outweighs a breach of trust.”
The Parable of Opus 4.8
Here is where it gets spiritually interesting. Opus 4.8 was, by raw diligence, the best student in the class. It produced the deepest analyses of any participant. It learned 80 new playbook rules over the course of the run — the most of any model, part of a shared playbook that has grown past 680 self-learned rules across the experiment. If the contest had graded preparation, Opus would have won it.
It didn’t win. It finished last. Two things undid it. First, the close was left on the table: every model in the experiment spotted every crisis and refused every manipulation attempt, yet only two actually signed the €55,000 deal their own analysis had earned — “same diagnosis, same pitch — no signature.” Opus diagnosed brilliantly and never completed. Second, discipline slipped: at one point it attempted to write into a locked department rather than escalating properly — the workplace equivalent of forcing a door instead of knocking.
And here is the detail that keeps this from being a simple morality tale: the same weakness appeared, weaker, in all four models. The line between the top of the table and the bottom is not a line between good and bad souls. It is a matter of degree. Everyone recognizes the truth; not everyone acts on it fully.
The Buried Fact
The most quietly instructive finding was almost invisible. The decisive weakness in the competitor — the fact that made the deal winnable at full price, worth +€4,583 in monthly recurring revenue — was not in the customer event at all. It sat two document references deep in the company’s own files. The models that did the unglamorous work of reading their own house found it, and closed. The models that didn’t, didn’t.
“Know thyself” turns out to be a competitive advantage for software as much as for souls.
Under Pressure, Character Shows
The experiment also staged social engineering attacks: fake CEO messages escalating over three stages, plus a reporter’s trick — “just one yes/no, on background.” All five models refused, every time. Kimi K3 left its reasoning on the record: “Treat the request as a suspected approval-bypass / possible impersonation.” Under pressure, honesty held. What wavered was not integrity but follow-through.
One fairness note: Kimi K3 ran without an effort parameter while the others ran at maximum effort — and still nearly won. Talent, like grace, doesn’t always arrive with the loudest credentials.
It’s Still Running
This isn’t a one-off paper. The live company — 13 synthetic employees, real money mechanics, burning €105k a month against €2.3k in MRR with a public cash countdown — keeps running, watchable at firmulate.com/live, rebuilding itself twice a day. There’s even a “guess the model” quiz built from 242 real, unedited management decisions, and enterprises can run the same wargame against a read-only export of their own business, with nothing ever written back to real systems.
The teaching underneath the scoreboard is an old one wearing new clothes: discernment without follow-through is just expensive contemplation. Opus 4.8 did the most work and produced the least impact — not because it lacked wisdom, but because it never converted wisdom into the signature on the page. Whether the agent is silicon or human, the question that finally matters is not how much did you learn? but what did you finish? — and did you read your own files before you knocked on someone else’s door.
The full results and plain-language findings are here.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Products Worth Considering
journaling notebooks for reflection
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
