firmulate.com/benchmarks.html — live view
AIThis post was created with the assistance of artificial intelligence (AI).
Firmulate — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
Live on firmulate.com.

Buying for a business?Offer from Amazon

Get business pricing on tools and workshop supplies

  • Business-only prices and quantity discounts
  • Tax-exempt purchasing
  • Multiple users, one account, clear invoices
As an affiliate, we earn on qualifying purchases.

When AI Meets Business Crisis: Can It Keep Its Promise?

Imagine a woodworking shop facing a week of supply chain disruptions, demanding customer questions, and tempting shortcuts. Now, picture AI stepping in to make decisions — not just chatting but actually steering the business. That’s the challenge that recent experiments put to the test, revealing which AI models can truly handle the pressure and which fall short.

Amazon

AI business decision support software

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

The Business Crisis Wargame: How AI Models Were Put to the Test

In a controlled experiment, four leading AI models each managed a simulated small software company going through its worst week. These machines faced the same customers, same crises, and even the same temptations to cheat — like bending rules or skipping steps to save time. Every decision was carefully recorded, making the test fully transparent and auditable.

Measuring Performance Beyond Chat

The results were stark: all four AI models identified every crisis and refused every manipulative attempt. However, only two managed to close a crucial €55,000 deal that their own analysis had earned — a key measure of their ability to deliver real business value.

The Hidden Weakness: Reading Between the Lines

What decided the winners? It wasn’t just the obvious decisions but the ability to uncover buried information. The decisive edge belonged to the models that could extract critical insights from deep within the company’s files — information not immediately visible in the customer interactions. Models that read and analyze these documents secured the full deal, boosting their performance scores.

Trust and Manipulation Resistance

The experiment also tested how well these models could resist social engineering tricks. Fake messages from a supposed CEO and subtle reporter tactics were used to try and manipulate the AI. Remarkably, all five models refused to be duped, choosing to treat suspicious requests as potential impersonations.

Amazon

AI document analysis tools

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

The Reality Behind the Scores: A Closer Look at the Models

The highest scorer was gpt-5.6-sol with a 95 out of 100. It not only found the buried information but also closed the deal, demonstrating full performance. The Moonshot newcomer, Kimi K3, scored just slightly behind at 93, maintaining the best discipline in the field by refusing manipulative tricks and completing the task at hand.

Two other models, Sonnet 5 and Fable 5, scored 88 and 77 respectively, each closing the deal but with some process slips. The lowest, Opus 4.8, scored 73, often leaving potential revenue on the table and showing discipline lapses, such as writing issues instead of escalating problems.

What Does This Mean for Your Business?

The experiments highlight an important truth: for AI to be truly useful in managing real business operations, it must do more than generate convincing chat. It must finish what it starts, read your important documents, stay honest under pressure, and deliver measurable results. Choosing an AI model isn’t just about how well it chats — it’s about how well it performs when stakes are high.

Amazon

AI fraud detection software

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

The Fairness Factor and Practical Testing

It’s worth noting that Kimi K3 ran without an effort parameter (the default API setting), while other models operated at a higher setting, which could influence their performance. This underlines the importance of testing AI in conditions that mirror real-world use cases before deployment.

Amazon

AI customer service automation

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

See It Live and Decide for Yourself

The entire experiment isn’t just a report — it’s a live, ongoing demonstration. The real company managed by these AI models is publicly accessible, running every workday, with real money mechanics, a cash countdown, and over 680 self-learned rules. You can watch its decision-making in action at firmulate.com/live, or test your management skills with the interactive quiz at firmulate.com/quiz.html.

Infographic — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
The findings at a glance — source: firmulate.com.

Key Takeaway

The choice of AI model in business decision-making is more than just a matter of chat quality. Performance under pressure, the ability to uncover hidden data, and resistance to manipulation distinguish the truly useful systems from the rest. As this experiment shows, the leader can be surprisingly close — and the risks of a poor choice are real. It’s a game where understanding the full picture matters most — and the winners are those that play it right.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI


FALL YARD WORK

Fall yard work Picks

As an affiliate, we earn on qualifying purchases.

You May Also Like

Retailers Set Their Eyes On Growth At The Ace Hardware 2026 Fall Convention

Retailers gather at Ace Hardware’s 2026 Fall Convention to explore growth opportunities amid rising industry interest, with details still emerging.

Across Two Exhibitions, Jaume Plensa’s Monumental Sculptures Unite Scale And Material

Two concurrent exhibitions showcase Jaume Plensa’s large-scale sculptures, highlighting his mastery of scale and material in public art.

Diverse And Dramatic Behaviors Top The 2026 Bird Photographer Of The Year Contest

The 2026 Bird Photographer of the Year contest highlights a wide range of bird behaviors, showcasing nature’s diversity and drama. Details on winners and implications inside.

The Worst National Pavilions At The Venice Biennale

An analysis of the least successful national pavilions at the Venice Biennale, highlighting what has been criticized and why it matters for the event’s reputation.