Predictive Analytics

“The Freakonomics of big data.”
—Stein Kretsinger, founding executive of Advertising.com; former lead analyst at Capital One

This book is easily understood by all readers. Rather than a “how to” for hands-on techies, the book entices lay-readers and experts alike by covering new case studies and the latest state-of-the-art techniques.

You have been predicted — by companies, governments, law enforcement, hospitals, and universities. Their computers say, “I knew you were going to do that!” These institutions are seizing upon the power to predict whether you’re going to click, buy, lie, or die.

Why? For good reason: predicting human behavior combats financial risk, fortifies healthcare, conquers spam, toughens crime fighting, and boosts sales.

How? Prediction is powered by the world’s most potent, booming unnatural resource: data. Accumulated in large part as the by-product of routine tasks, data is the unsalted, flavorless residue deposited en masse as organizations churn away. Surprise! This heap of refuse is a gold mine. Big data embodies an extraordinary wealth of experience from which to learn.

Predictive analytics unleashes the power of data. With this technology, the computer literally learns from data how to predict the future behavior of individuals. Perfect prediction is not possible, but putting odds on the future — lifting a bit of the fog off our hazy view of tomorrow — means pay dirt.

In this rich, entertaining primer, former Columbia University professor and Predictive Analytics World founder Eric Siegel reveals the power and perils of prediction:

• What type of mortgage risk Chase Bank predicted before the recession.
• Predicting which people will drop out of school, cancel a subscription, or get divorced before they are even aware of it themselves.
• Why early retirement decreases life expectancy and vegetarians miss fewer flights.
• Five reasons why organizations predict death, including one health insurance company.
• How U.S. Bank, European wireless carrier Telenor, and Obama’s 2012 campaign calculated the way to most strongly influence each individual.
• How IBM’s Watson computer used predictive modeling to answer questions and beat the human champs on TV’s Jeopardy!
• How companies ascertain untold, private truths — how Target figures out you’re pregnant and Hewlett-Packard deduces you’re about to quit your job.
• How judges and parole boards rely on crime-predicting computers to decide who stays in prison and who goes free.
• What’s predicted by the BBC, Citibank, ConEd, Facebook, Ford, Google, IBM, the IRS, Match.com, MTV, Netflix, Pandora, PayPal, Pfizer, and Wikipedia.

A truly omnipresent science, predictive analytics affects everyone, every day. Although largely unseen, it drives millions of decisions, determining whom to call, mail, investigate, incarcerate, set up on a date, or medicate.

Predictive analytics transcends human perception. This book’s final chapter answers the riddle: What often happens to you that cannot be witnessed, and that you can’t even be sure has happened afterward — but that can be predicted in advance?

Whether you are a consumer of it — or consumed by it — get a handle on the power of Predictive Analytics.

Q & A with Author Eric Siegel

Why does early retirement decrease life expectancy and why do vegetarians miss fewer flights?

These are two more colorful examples of the multitudes of predictive discoveries waiting within data.

University of Zurich discovered that, for a certain working category of males in Austria, each additional year of early retirement decreases life expectancy by 1.8 months. They conjecture that this could be due to unhealthy habits such as smoking and drinking following retirement.

One airline discovered that customers who preorder a vegetarian meal are more likely to make their flight, with the interpretation that knowledge of a personalized or specific meal awaiting the customer provides an incentive, or establishes a sense of commitment.

Predictive analytics seeks out such predictive connections and then works to see how they may combine together for more precise prediction.

What are the hottest trends in predictive analytics?

There have been many exciting improvements in the core technology of predictive analytics. One is “uplift modeling” (a.k.a. “persuasion modeling”), which predicts influence . . . in order to do influence. The Obama campaign used it to influence voters in the 2012 presidential election; marketing uses it to more adeptly persuade customers; and medicine uses it to better select per-patient treatments. This topic is the focus of the final chapter of this book.

Another hot trend is ensemble models. Like the collective intelligence that spawns the wisdom of a crowd of people, we see the same effect with a crowd of predictive models. Each model alone may be fairly primitive such as a few simple rules, so it gets prediction wrong a lot, as an individual person trying to predict also does. But have them come together as a group and there emerges a new level of predictive performance.

Did Nate Silver use predictive analytics to forecast Obama’s election?

No–but Obama did. Nate Silver made election forecasts for each state as a whole: which way would a state trend, overall? In the meantime, the Obama campaign was using predictive analytics to make per-voter prediction. Moving beyond forecasting, true power comes in influencing the future rather than speculating on it–the raison d’être of predictive analytics. Nate Silver publicly competed to win election forecasting, while Obama’s analytics team quietly competed to win the election itself. Specifically, team Obama drove per-voter campaign decisions by way of per-vote predictions.

What is the coolest thing predictive analytics has done?

One of the most inspiration accomplishments of predictive analytics is IBM’s Watson, which was able to compete against the all-time human champions on the TV quiz show Jeopardy! The questions can be about most any topic, are intended for humans to answer, and can be complex grammatically. It turns out that predictive modeling is the way in which Watson succeeds in determining the answer to a question: it predicts, “Is this candidate answer the correct answer to this question?” It knocks off one correct answer after another–incredible.

What are companies predicting about me as a customer?

Here are just a few examples:

• Microsoft helped develop technology that, based on GPS data, accurately predicts one’s location up to multiple years beforehand.
• Target predicts customer pregnancy from shopping behavior, thus identifying prospects to contact with offers related to the needs of a newborn’s parents.
• Tesco (UK) annually issues 100 million personalized coupons at grocery cash registers across 13 countries. Predictive analytics increased redemption rates by a factor of 3.6.
• Netflix sponsored a $1 million competition to predict which movies you will like in order to improve movie recommendations.
• One top-five U.S. health insurance company predicts the likelihood an elderly insurance policy holder will die within 18 months in order to trigger end-of-life counseling.
• Con Edison predicts energy distribution cable failure, updating risk levels that are displayed on operators’ screens three times an hour in New York City.

Table of Contents

Chapter 1: Liftoff! Prediction Takes Action
Chapter 2: With Power Comes Responsibility: Hewlett-Packard, Target, and the Police Deduce Your Secrets
Chapter 3: The Data Effect: A Glut at the End of the Rainbow
Chapter 4: The Machine That Learns: A Look Inside Chase’s Prediction of Mortgage Risk
Chapter 5: The Ensemble Effect: Netflix, Crowdsourcing, and Supercharging Prediction
Chapter 6: Watson and the Jeopardy! Challenge
Chapter 7: Persuasion by the Numbers: How Telenor, U.S. Bank, and the Obama Campaign Engineered Influence