Expounding on the "Human irrationality" theme in Daniel Kahneman's Thinking, Fast and Slow, a WSJ article explores the notion that algorithms will (essentially) save us from our own bad (biased) decision-making:
"Computer systems are now becoming powerful enough, and subtle enough, to help us reduce human biases from our decision-making. And this is a key: They can do it in real-time. Inevitably, that "objective observer" will be a kind of organic, evolving database.
These systems can now chew through billions of bits of data, analyze them via self-learning algorithms, and package the insights for immediate use. Neither we nor the computers are perfect, but in tandem, we might neutralize our biased, intuitive failings when we price a car, prescribe a medicine, or deploy a sales force. This is playing "Moneyball" at life."
Of course, there is one small problem. There is an overwhelming shortage of qualified data scientists to devise and infer actions (or non-actions) from these (decider) algorithms.
The data scientist talent shortage is forcing organizations to ship their data to the processing talent, somewhat in conflict with the big data premise "moving computation is cheaper than moving data". But, in the end, business-technology decisions hinge on value, rather than technical principles.
One company realizing value from an analytics-as-a-service provider is Schwan's Home Delivery:
"For a glimpse, look inside The Schwan Food Co., whose 6,000 roving sales people deliver frozen products to homes of three million customers across the country.
Schwan home sales were listless for four straight years, beset by high customer churn and inventory pileups. Over 10 months, the venerable Minnesota company began a program with the aid of Opera Solutions Inc. of New York, an eight-year-old analytics firm.
Schwan already had a crude recommendation program. Its sales people could look at six weeks of orders, and suggest purchases from that list.
The new project took it into more sophisticated territory: Matching seemingly disparate customers with similar purchase patterns in their past. Opera calls them finding "genetic twins." It also added ways to track whether customers' spending was fading from certain categories—say, breakfast foods—and offered product suggestions and discounts to keep the spending intact.
Schwan's database is now pushing out more than 1.2 million dynamically-generated customer recommendations every day, sent directly to drivers' handheld devices. Opera says Schwan's revenues are up 3% to 4% because of it."
Schwan's isn't alone. The demand for data science (and scientist) centered technology service organizations continues to grow. 1010data just reported it hosts 5 trillion records for its customers, all business records.
And most recently, and perhaps tellingly, rumors abound that Amazon will be adding analytics-as-a-service to its offerings.
Quite possibly, we will find ourselves in a "there's an algorithm to decide that" world. But, until the talent shortage is stemmed, we'll need to get our rationality delivered. Though, odds are, most won't trust it.
