My goal in this blog is to bring together on web/paper an outline of ways that I see that decision and data sciences are better together than apart. And by better, I mean more relevant to top level decision makers, executives and the C-suite, at their respective companies or non-profits.
While in graduate school my research was in the area of decision making under uncertainty. The types of decisions concerned were really big decisions, decisions that were so expensive to undo that it was often worth the cost to build in flexibility in order to handle uncertain futures. My applications were in the area of capital intensive product development; the decision theory used in these scenarios is the same decision theory that can be applied to any important decision. The goal of these decisions was to maximize discounted utility, often in the form of discount cash flows, under exogenous and endogenous uncertainties.
In the time before ‘big data’ these decisions were often made with little data, and the cost of making these decisions was expensive. Therefor the techniques were utilized only for big decisions. In addition, that little data was sometimes to questionable to use for decision making, owing to uncertain futures and that the past does not necessarily predict the future. There were many techniques developed and implemented from decision analysis: decision trees, influence diagrams, utility functions, subjective probability (aka, eliciting probabilities from experts), multi-criteria decision analysis, etc. The heroes in this journey were Ronald Howard, Howard Raiffa, Ralph Keeney, and others.
[Aside: One of my favorite cautionary tales of using predictions is the history of the estimate of the speed of light (a believed to be universal constant). Here one sees that confidence interval estimates as late as 1940 for the speed of light did not overlap with what is now believed to be the universal value.]
Today, for some decisions, we have copious amounts of data, enough data even to make ‘accurate’ predictions and probability estimates of the outcome. This is the realm of predictive analytics and its tools are statistical and machine learning algorithms. Common industrial and non-profit questions to ask are ‘To what extent is a customer/donor at risk of leaving an offering?’ (customer churn) and ‘If I take an action, what is the probability that the target person will purchase/donate?’. Because we have data on lots of previous examples of interventions and their results, we build models and use those models on unseen cases to make probability estimates.
These predictions still need to be used for decision making and this is where decision analysis comes to play. Decision analysis allows us to answer ‘given the probability of this customer leaving, the probability of them responding to this intervention, the cost of the intervention, and the expected benefit if the intervention is successful’ is it profitable to take this intervention for this customer? Now that the cost of making that decision is low due to the data at hand and the strength of machine learning models, that sort of decision can be made at a customer by customer level. At a higher level and combining predictive analytics and decision analysis techniques, we can answer the more strategic questions such as ‘should we invest in a customer retention program? and if yes, at what level.’ Provost and Fawcett do a very good job of explaining how one goes about using decision analysis and data science to answer these sorts of questions. At a more technical level Slater Stitch provides a good example of making these types of decisions, combining machine learning and decision analysis.
In the research arena, MIT’s Prediction Analysis Lab, Cynthia Rudin and other researchers are working on combining machine learning and decision making. In my prior experience I worked in combining methods from operations research (constrained optimization problems) with machine learning by using the predictions from machine learning as forecast inputs into the decision (aka optimization) problem. My particular application for optimizing asset utilization (trucks, people, other equipment), used forecasts of demand (which could often change within a day, sometimes leading to the optimal decision not to fully allocate all resources and let the day unfold). Rubin has taken this a step further than the two step linear process: their research used the outcome of the optimization problem as feedback to the predictive modeling step, machine learning for robust optimization, and have established a (mathematical) connection between machine learning and classic operation research problems (papers are reverenced below).
Data science and decision science / operations research are linked. INFORMS, The professional society for Operations Research and Management Science, sensed this when they added analytics to their charter. The notion that it is one vs the other, or that one begins where the other ends, is narrow and ignores the shared history in common scientific and mathematical domains. Rudin and other researchers are demonstrating methods for systems optimization combining data and decision sciences. Data scientists and decision analysts who embrace this merger will strategically align their work to the goals of their businesses/non-profits. Maybe we need a new title to describe the people that do these super-human activities: rational decision scientist (no one has ever invited me to work in branding) or cyborg decision analyst. In any case, it’s my experience and opinion that the combined forces of data science and decision analysis are greater than the sum of their parts.
Speed of light measurements graph courtesy of http://micro.magnet.fsu.edu/primer/lightandcolor/speedoflight.html
T. Tulabandhula and C. Rudin. Machine learning with operational costs. Journal of Machine Learning Research, 14:1989–2028, 2013.
T. Tulabandhula and C. Rudin. Generalization bounds for learning with linear, polygonal, quadratic and conic side knowledge. Machine Learning, pages 1–34, 2014.
T. Tulabandhula and C. Rudin. On combining machine learning with decision making. Machine Learning, 93:33– 64, 2014.
T. Tulabandhula and C. Rudin. Robust optimization using machine learning for uncertainty sets. In Proceedings of the International Symposium on Artificial Intelligence and Mathematics (ISAIM), 2014.