Deductive vs. Inductive Approaches in Analytics

---

You may be wondering, OK, where is he going with this post? Bear with me and let me explain. First a quick review of the two constructs. Deductive thinking is when you start with a theory, develop a null hypothesis, gather data through observations, and then disprove your null hypothesis. For example, you could have a theory that cold weather causes ice cream sales to decline. So, you would form a null hypothesis (weather does not affect sales), gather data, and attempt to disprove that null hypothesis. Mathematics, and its theorems, is the best example of deductive thinking. The theorems are true, even when combined or extended into other contexts.

In contrast, inductive thinking starts with the data, it involves collecting data first, examining trends, forming a tentative hypothesis, and then finding data to support the hypothesis. For example, you start out examining POS data combined with weather data and you notice a drop in ice cream sales during a cold snap in the southeast U.S. So, you can form a hypothesis that weather affects ice cream sales and then gather more data to support (not confirm) that hypothesis. The important thing to note is that you did not start out with any preconceived theory, the examination of the data led you to form some conclusions.

Taleb (2010) in his seminal book, The Black Swan: The Impact of the Highly Improbable provides a good example of the problem with inductive thinking. He states that people for hundreds of years thought all swans were white, as that is what the data was showing (i.e., observing swans at the lake). That "theory" was dismantled in an instant when a black swan was discovered in remote Australia in the late seventeenth century. The lesson is that the data you are examining may in fact support some hypothesis, but you may not have all the data -- you could be just seeing a small part of the reality of the situation. He warns that the world is incredibly complex, so be humble when analyzing or predicting an entity, especially one created by human beings (more on this in a later post).

There is no wrong approach in analytics, both deductive and inductive thinking are used every day in our lives. However, in the business realm it is more common to leverage inductive thinking as we are trained to analyze mountains of empirical data to attempt to spot trends. In addition, many times we are examining cross-sections of time-series data aggregated across various dimensions such as geography, store and, category. This analysis can lead us to draw some hypothesis about what is really happening with our product in the marketplace. And in many cases, these suppositions may be true in the aggregate; however, can start to fall apart when looking at other deeper slices of the data.

So why this post on this topic? Well, in my 25+ years of analyzing business operations, I have developed a healthy dose of humility about what aggregated data can tell one about their business. We are attempting to explain complex human behavior aggregated across millions of data points, and any cause-and-effect propositions are specious at best. So, while it's necessary and valuable to leverage inductive thinking when analyzing empirical data, it's also very easy to make the leap to -- it's always true. In a modern quote based on Aristotle's writings, "The more you know, the more you realize you don't know."

JimSwallow@AnalyzeBrand.com

References:

Aristotle. The Metaphysics.

Taleb, N. N. (2010). The Black Swan: The Impact of the Highly Improbable: With a new section:" On Robustness and Fragility" (Vol. 2). Random house trade paperbacks.