Modeling Human Behavior and The Rise of Big Data

Published November 27, 2013   |   
Will Craft

In the wake of the 2012 Presidential election, hundreds of Internet users asked a very important question: “Is Nate Silver a witch?” The accusations of witchcraft came after Silver, statistician and blogger for the New York Times, predicted the winner of all 50 states with 100% accuracy [1]. Following his perfect prediction of the outcome of all 38 Senate races and correctly calling 49 out of 50 states in the 2008 presidential election cycle, this victory for the power of statistics has lead many to declare that the age of Big Data has arrived [2].

The age of Big Data refers to the increasing tendency for scientists and statisticians to utilize vast amounts of data to computationally model and predict human behavior. Data is drawn from an array of sources ranging from the chatter of the Twitterverse to the meticulously gathered databases of academic and government organizations. Analysis that utilizes big data is a shift away from theories and models based on a small number of case studies, towards the usage of as much data as possible to try and tease out patterns of human behavior. Elections are well suited to statistical analysis and have long been a target of prediction. The mantra of Big Data is that all human behavior is a subject for creative and innovative statistical analysis. The scientists of this new age examine correlation and create statistical models to predict and analyze everything from human voting patterns to—even more interestingly—musical creativity and human intelligence.

The idea behind this new trend is that if we can gather enough data, a sufficiently advanced statistical model will allow scientists and statisticians to tease out an explanation of human behavior from the numbers that we leave in the wake of our everyday lives. The hope is that a complex understanding of humans, from patterns of behavior to creativity, is waiting to be uncovered.

Read More