Why Does AI Fail in New Markets?

Artificial Intelligence has had brilliant success in some markets.

Unlike prior peaks of false AI hope, real value is being created. Inexpensive computers can now operate large (or “deep”) neural networks and perform some tasks which used to require humans.

But successful application has been uneven.

During 2017 several high-profile proof-of-concept trials failed. Those in the business know of many more quiet failures. Some failures were stunning. Teams who had been successful in one market, backed with large war chests saw projects crash and burn.

This leads some to wonder if AI winter is approaching again. We don’t share this view; there are too many applications where returns on AI investments are great.

On the other hand, much of AI is immature and hype filled. As part of a three-year benchmarking program on modeling, simulation and analysis, we surveyed and interviewed hundreds of people in dozens of firms. No AI or Machine Learning practitioners were best practice exemplars.

We developed an AI/ML technology and math risk checklist. We think it has real value. But it does not tell us why successful AI and ML teams fail when they move to new markets.

We think there is a softer, non-geeky reason: culture.

The pictures show Deal and Kennedy’s map of cultures. In the 1980’s they found they could predict an organization’s norms by asking two questions:

1. How big are the consequences of a bad decision?
2. How long does it take to find out if a decision was bad?

They gave names to cultures based on the four possible combinations of answers.

The Tough Guy culture faces big consequences and gets rapid feedback. These are the first responders, the special forces, high speed traders. They are exactly opposite the Process Driven culture where feedback takes a long time, and consequences are small. The bureaucrats have nothing in common with the tough guys.

Most successful AI and ML applications backed with venture funds have come from web and advertising markets. They get fast feedback, and consequences of errors are small. When 99.99% of advertisements of product recommendations are wrong without AI, you are 100 times better at 99% wrong.

In the Work Hard Play Hard world of Silicon Valley, being “less wrong” generates billions of dollars of value. But now those teams hope to serve cultures called “Bet the Company.” Industrial firms, energy firms, transportation firms all make large, long term bets on expensive capital assets. They want to be right, not less wrong. They need evidence they are on the right path, not unexplainable AI.

The Silicon Valley mindset is on a different planet. Their software based businesses avoid owning hard assets. They don’t “get” the long-time horizon perspective of Bet Your Company cultures. These two cultures will never agree on how to think about ideas like “failing fast.”

Everyone agrees there are huge gains to be made in firms with Bet the Company cultures. These giants have value trapped between their silos. They need better insights and more foresights. They aren’t getting it from most AI and ML trials at their operational core. There are successes in things like help desks and training. But these are not the operational core.

There are forms of analytics and machine learning which meet the core business and cultural needs of Bet the Company organizations. They aren’t the methods used to place banner ads or make recommendations about which hair conditioner goes best with your shampoo.

Until the very real differences in user needs, culture and math are more widely understood, we will see bright teams from retail and advertising fail in their efforts with Industrial giants, energy firms, and transportation. That explains many of the failures of AI in these new markets.

And, it helps explain why Lone Star^® is succeeding.