American Association for Physician Leadership

When an Educated Guess Beats Data Analysis

Oguz A. Acar | Douglas West

August 18, 2021


Summary:

Heuristics and gut feelings often offered a better tradeoff in terms of decision-making speed and accuracy; the inclusion of analysis in the decision-making process did not bring about any meaningful improvement in accuracy while significantly reducing speed.





Research on data provided by 122 companies in the advertising, digital, publishing, and software sectors (industries characterized by uncertainty over outcomes) suggests that data driven decision-making could be counter-productive under conditions of uncertainty. Heuristics and gut feelings often offered a better tradeoff in terms of decision-making speed and accuracy; the inclusion of analysis in the decision-making process did not bring about any meaningful improvement in accuracy while significantly reducing speed.

Data-driven decision making is often viewed as the gold standard in modern management. And this is for good reason. The explosion of available data and rapid advances in data science enable managers to know substantially more about their business. This knowledge, if used well, should bring about better decision-making on about every aspect of the business.

This is perhaps why most companies are in a horse race in building analytical capabilities to make the most of this unprecedented abundance of data. For example, a recent survey of Fortune 1000 companies shows that 91.9% of firms report increasing investment in data initiatives.

While the potential of big data is irrefutable, is it the panacea for all decision-making situations? Put differently, could a strong emphasis on data and analysis backfire under some circumstances? We explored this in our recent research.

Our intuition was that data-driven decision making could be counterproductive under extreme uncertainty. In such cases, it will be highly challenging and sometimes impossible to collect reliable data. This could explain why 12 publishers were unable to see the potential of Harry Potter and the Philosopher Stone before Bloomsbury Publishing accepted to publish an initial print run of 500 copies. The book was so innovative that there was by definition no prior data available to accurately assess its potential.

To test our intuition, we collected data from 122 companies in creative industries (in advertising, digital, publishing, and software sectors) about their latest innovation projects. We chose creative industries due to high levels of uncertainty about customer reactions and an infinite variety of potential new products and product modifications. For the same reason, we focused on innovation screening decisions — the decision to select what innovation projects to pursue for development. These decisions are characterized by high uncertainty; managers often lack sufficient past data that would enable them to predict customer reactions accurately, market potential, feasibility, and risks. Even if they had such data, it would often be extremely difficult and sometimes even misleading to extrapolate.

We asked managers in these companies to think about their most recent innovation project for which they needed to make a screening decision and included questions to understand how they made this decision. Specifically, the questions addressed the extent to which they relied on analysis (i.e., choosing the option that proved best based upon analyzing the data), instinct (i.e., choosing the option following their instincts), and a range of well-known heuristics (i.e., practical strategies to make decisions faster and more frugally). These heuristics included “tallying” (choosing the option with highest number of favorable points), “experience” (choosing the option most experienced person in team wanted) and “majority” (choosing the option most people wanted) amongst several others. Next, we asked managers to indicate whether they think they got the decision right (perceived decision-making accuracy) and how fast they were in making the decision (perceived decision-making speed).

The results first showed, to our surprise, despite the huge interest in big data, that the managers in our sample did not rely on analysis any more than on their instincts or some of the simple heuristics. The most commonly used heuristic, more than both analysis and instinct, was tallying.

We also find that relying on analysis is not necessarily the ideal way to choose between innovation projects. While the decisions based on data analysis brought about a good level of decision-making accuracy, the process was slow. Managers who relied on their instincts together with some simple heuristics made decisions that were just as accurate but were undertaken much more quickly. That is, heuristics and gut feelings offered a better tradeoff in terms of decision-making speed and accuracy; the inclusion of analysis in the decision-making process did not bring about any meaningful improvement in accuracy while significantly reducing speed.

One note of caution for managers who consider embarking on gut-based innovation decisions: the effectiveness of their intuition might rely on prior experience. Prior research suggests that the effectiveness of intuition compared to analysis is contingent upon domain knowledge; experts in a domain are more likely to make better gut decisions. Managers with limited domain expertise might therefore be better off by refraining from extensive reliance on intuition. Our results suggest relying mainly on heuristic also presents a viable alternative.

The next time when you face a managerial decision that is ambiguous, bear in mind that data might not be the only basis for a choice. Following your instincts, together with some simple heuristics, can lead to quicker and potentially as accurate decisions especially for those with the requisite expertise.

Oguz A. Acar is an associate professor (reader) at City University of London’s Bayes Business School (formerly Cass) and a fellow of the Royal Society of Arts.

Douglas West is a Professor in Marketing at King’s Business School, King’s College London in England.

Copyright 2021 Harvard Business School Publishing Corporation. Distributed by The New York Times Syndicate.


Oguz A. Acar

Oguz A. Acar is an associate professor (reader) at City University of London’s Bayes Business School (formerly Cass) and a fellow of the Royal Society of Arts.


Douglas West

Douglas West is a Professor in Marketing at King’s Business School, King’s College London in England.

Interested in sharing leadership insights? Contribute


Topics


Related

Recommended Reading


For over 45 years.

The American Association for Physician Leadership has helped physicians develop their leadership skills through education, career development, thought leadership and community building.

The American Association for Physician Leadership (AAPL) changed its name from the American College of Physician Executives (ACPE) in 2014. We may have changed our name, but we are the same organization that has been serving physician leaders since 1975.

CONTACT US

Mail Processing Address
PO Box 96503 I BMB 97493
Washington, DC 20090-6503

Payment Remittance Address
PO Box 745725
Atlanta, GA 30374-5725
(800) 562-8088
(813) 287-8993 Fax
customerservice@physicianleaders.org

CONNECT WITH US

LOOKING TO ENGAGE YOUR STAFF?

AAPL providers leadership development programs designed to retain valuable team members and improve patient outcomes.

American Association for Physician Leadership®

formerly known as the American College of Physician Executives (ACPE)