The rise of decision intelligence

Preface

Entelect is a silver sponsor of the annual ITWeb Business Intelligence 2023 Summit, to be held from 7-8 March.

The importance of leveraging data for decision-making has become even more critical over the last two years. But is simply having access to good data enough to make good decisions?

Join us at ITWeb Business Intelligence Summit 2023, where we will unpack transitioning from business intelligence to decision intelligence with Erwin Bisschops, data solutions practice lead at Entelect.

Get a preview of Bisschops' talk below, as he discusses the rise of decision intelligence and what it means for business intelligence in the future.

The evolution of analytics has seen many different terms being coined over the last couple of decades, from management information systems and decision support systems to business intelligence (BI) and now, the new kid on the block, decision intelligence (DI). Gartner named DI a top trend for 2022, and 2023 will be no different. It also put a clear definition in place (shared a little later), which transforms DI from a vague marketing trend to an increasingly important business strategy.

Let's have a closer look at what happened with BI over the years and whether DI is here to replace it.

We've all seen the many promised benefits surrounding BI, like:

• Use data visualisation to make better decisions, faster;
• Self-service BI will bring data democratisation, which in turn will bring improved productivity, improved decision-making and knowledge optimisation; and
• BI will give you actionable intelligence.

The intent of BI has always been to solve a common problem: how can organisations benefit from analysing a mountain of data without being an expert in query languages, database technology and statistics? Along the way, users of BI solutions bumped into various problems, like data quality issues, data availability issues and hard to understand database structures, to name a few.

A lot of focus was placed on the functionality of BI tools themselves; it often was a race between vendors for being the most complete, easiest-to-use, fastest, most flexible and most cost-efficient tool. In recent years, it seems that Microsoft's Power BI has risen to the surface as the strongest contender in this arena. However, the real challenges within BI have always been overshadowed by the technical prowess and capabilities of BI tools and platforms. Technically inclined people often dominated BI discussions with a focus on these features and the infrastructure required, rather than spending enough time on the actual business decision-making requirements. The means instead of the end.

With the current aspirations for so many organisations to become data-driven and adopt corresponding cultures and tools, it is important to spend enough time on the topic of decision-making itself.

After all, the information and insights provided by numerous dashboards and reports don't necessarily lead to better decisions. The brain of the consumer of the information is still the place where the actual decision-making happens, and this person also implements those decisions. Over the years, organisations have ended up spending huge amounts of money on analytics related hardware, cloud services, tool training and implementation projects, only to experience varying degrees of disappointment that those business benefits never materialised.

Figure 1: The human brain – the final frontier for BI.

---

A Bain & Company survey of almost 800 companies from markets around the world shows that high-quality decision-making and strong performance go together. This survey highlights a 95% correlation between companies that excel at making and implementing key decisions and those with best-in-class financial results. It is worthwhile to mention that decision-making can get rated across three elements: quality, speed and implementation. (Marcia W. Blenko,

)

Unfortunately, the link between investments in BI environments and improved decision-making is weak:

• On average, organisations only use 50% of available information for decision-making;
• Only 24% of executives make decisions based on data (BARC, global survey on data-driven decision-making in businesses, n=728); and
• More than half of Americans rely on their gut to decide, even when faced with contrary evidence (Ohio State University).

Romanticising gut-based decisions is in part caused by the geniuses of recent times. Albert Einstein is often quoted saying: "The intuitive mind is a sacred gift" and Steve Jobs is famous for saying: "Have the courage to follow your heart and intuition; they somehow already know what you want to become." We know that intuition can be a very helpful tool, but it's a mistake to base most business decisions just on gut feel. From an evolutionary perspective, gut feel was important to help us survive in a hostile environment as hunter/gatherers, but it wasn't designed for decision-making in a complex business environment.

In his book: "Thinking, slow and fast", Nobel prize winner Daniel Kahneman explains how our brain has two operating systems. System one is fast, unconscious, automatic, effortless and does 95% of all our thinking. System two is slow, deliberate and conscious, effortful, rational and only does 5% of all our thinking. We humans love to conserve energy and take short-cuts – we are satisficers (https://en.wikipedia.org/wiki/Satisficing) and corner-cutters when it comes to decisions. Kahneman's research points out that system two is lazy and essentially a slave to our system one. System one sends suggestions to our system two, which then turns them into firm beliefs.

Figure 2: Our brain and decision-making (source: Kahneman, 2011).

---

Analysis is also hampered by what is called the theory of small numbers. We tend to see relationships in the data and, even more importantly, causality, which most often is not there. Events around us are more random than we'd like to think they are. When someone gets a valuable insight from BI, it might not be true simply because of the amount of underlying data points not being large enough. Therefore, the business decisions made based on this initial insight will likely not lead to the expected results, which in turn leads to disappointment in the BI environment.

An example of the theory of small numbers predicament is the Gates Foundation's research into the characteristics of the most successful schools. The study found that smaller schools were more successful and huge investments have since been made in creating smaller schools. However, the problem with these findings was that the underlying sample sizes were way too small. The truth is that small schools are not better on average; they are simply more variable. If the analysts who reported to the Gates Foundation had asked about the characteristics of the worst schools, they would have found that bad schools also tend to be smaller than average.

We live in a VUCA world (volatile, uncertain, complex, ambiguous) and business decision-makers are often faced with an increase in number and complexity of decisions to be made. Think of disruptive supply chain forces, increased employee churn, rapidly changing marketplaces and the rise of the digital shelf, to name a few. Making the right decisions has become so much more challenging in recent years. And… self-service BI is not sufficient to come up with the answers we need.

Decision Intelligence intends to provide the decision-making outcomes that BI couldn't deliver. Gartner's definition of DI is: "A practical domain framing a wide range of decision-making techniques bringing multiple traditional and advanced disciplines together to design, model, align, execute, monitor and tune decision models and processes. Those disciplines include decision management (including advanced nondeterministic techniques such as agent-based systems) and decision support as well as techniques such as descriptive, diagnostics and predictive analytics."

Decision intelligence can also have a strong focus on the applied aspects of data science and aims to build processes for getting from the conception of an idea to a data science project running reliably, (partially) automating decision-making steps where possible. It aims to move away from thinking that you're data-driven (you make the decision you wanted to make all along based on your unconscious biases and not based on the data itself – Kahneman's system one in action), to truly data-driven decision-making (Kahneman's system two). Therefore, typical data approaches will have to be combined with behavioural and managerial sciences.

The latter implies that decision sciences also form part of DI, like economics, decision analysis, psychology, neuroeconomics, behavioural economics, experimental game theory, design, ux research and philosophy. Hence the inclusion of Kahneman's research in this article for an initial context to the impact of behavioural sciences on decision-making. As such, DI is an amalgamation of quantitative (applied data science and statistics) and qualitative aspects (social and managerial sciences).

The business benefits to be derived from DI are numerous: making more decisions in real-time, improving customer service, improving top line and achieving cost savings, start making automated decisions that were previously out of reach. An example of the latter is to connect media and promotions planning decisions to supply chain decisions – when you run a promotion, you will know beforehand that the product you're advertising will be physically available in regions where the ad will be seen. The focus is very much on connecting the dots that are currently not connected.

DI will also help with employee satisfaction. The Great Resignation has left a lot of organisations with less staff and, in some cases, less skills and lower experience levels. DI can fill this gap, giving employees an increased sense of accomplishment in their jobs by providing them with a systematic and substantiated approach.

BI and DI are not at loggerheads with each other and I can see an analytical landscape where the two happily co-exist. If we look at figure 3 below of a typical decision-making process, we can see that step 1 and 2 can be perfectly accomplished by using typical BI functionality. Step 3 will then be taken care of by DI. Reviewing your decision (which can also be fulfilled by BI, provided the information is available) can lead to the identification of new decisions to be made, hence the continuous cycle.

Figure 3: The decision-making process.

---

For traditional reporting and dashboarding requirements that address common business-as-usual situations, BI will remain a superb tool. Self-service BI functionality is great for business areas that are in a so-called discovery mode, especially diagnostic reporting (as opposed to descriptive reporting) is greatly dependent on having that data at your fingertips. However, self-service BI is also the area where BI has the highest chance of failure due to its dependency on opaque, and oftentimes biased, reasoning and decision-making processes by the human brain. This is an area where DI can help by creating transparency.

Decision intelligence is one of the trends to watch in the data analytics space this year. DI lends itself perfectly for the (partial or complete) automation of decision-making processes, which need to be guided by a well-defined business decision to be made. DI will not replace the role that BI plays, but, as we can see above, DI and BI will work closely together in a decision-making landscape.