Topic Tuesday #47 2013/06/11 - "Big Brother / Big Data"
Orwell would be pointing a malnourished finger at all of us and chanting, "I told you so".I don't go into the dystopian conspiracy theories, but as they are part of our culture, they still must be examined. Today the magnifying glass is on "Big Data". You may have heard the term, and if you haven't, you will.
Wikipedia summarized it thusly:
Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. The challenges include capture, curation, storage, search, sharing, transfer, analysis, and visualization. The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data, as compared to separate smaller sets with the same total amount of data, allowing correlations to be found to "spot business trends, determine the quality of research, prevent diseases, link legal citations, combat crime, and determine real-time roadway traffic conditions.
Big Data is just that, BIG. Veritable truckloads of data available on demand and manipulatable to yield a variety of correlations. It's enough to give you the heebie-jeebies, but honestly, it is unavoidable.
Big Data is a side effect of our increasingly technological society. We have devices that generate information that can be captured and logged. Most of it is innocuous. Like temperatures, wind speed, and rainfall.
We take weather measurements every few seconds (this would be a data set, like a spreadsheet) in thousands of weather stations all over the world (a larger data group, a collection of spreadsheets). Now imagine that you have all this information collected from all the weather stations all over, and now you can see patterns. With patterns you can make predictions. Voila, you have a rudimentary weather model and can start to predict storm patterns.
Now extrapolate that out further. Do you have a credit card? Congratulations you have your own data set of purchasing patterns! This information is stored and used to determine fraud patterns. If suddenly you are outside your normal spending patterns or regions, you may be flagged with a fraud alert, keeping you safe. The dark side of the credit card industry is they have a tendency to sell/share that information with marketers and even law enforcement. In this way your habits become a recognizable pattern. Patterns can be identified, and some are as unique as a finger print.
It is safe to assume that if your have a device that generates a loggable data set, you can be sure someone somewhere is collecting it, and someone else wants it for some reason. Some will want to make life easier for you, others for themselves. Some will profit from it, and others will suffer. And I haven't even got into facial recognition!
