A bit of a dry post today, unless you're a data geek like we are.

Since our launch, we've entered a few customer and partner conversations that would've been best served with an introductory primer in data mining.  I would estimate that a majority of marketers don't have a firm grasp on the requirements and/or what the opportunities are in the field of data & data-driven decision making.

So here's my "business guy shorthand" on data mining, what's available today, and the opportunities all of this affords.

First of all, content on the Internet has exploded as others have chronicled over the last few years. From the linked article above: 

"…compliments of Dave Turek, the guy in charge of supercomputer development at IBM: From the year 2003 and working backwards to the beginning of human history, we generated, according to IBM’s calculations, five exabytes–that’s five billion gigabytes–of information. By last year, we were cranking out that much data every two days. By next year, predicts Turek, we’ll be doing it every 10 minutes."

How did this happen, you ask?  Look no further than the social media movement, which democratized publishing to the web via blogs, put our videos online via YouTube and Vimeo, broadcasted our short-form opinions via Twitter, and chronicled our life through Facebook.  And alongside those services are a series of other services that all collectively help us chronicle our lives online.

I personally love this infographic below that illustrates just how much data we're all creating today.  It's a far cry from when we needed a web developer to post content for us.  Additionally, check out this compendium of stats from 2011.

In addition to the revolution in self-publishing, we've simultaneously had a revolution in the collection of data in the corporate realm.  Gone are the days of Day Timers, Day Minders, etc.  In with the world of Salesforce.com.  New Point of Sales (POS) systems are no longer analog but rather "digital", which allow for the collection much more data that can be analyzed and explained. Every part of a modern business can be digitized and recorded today for data analysis — and smart businesses are doing just that even if they don't have the analysis capability just yet.

Additionally, government agencies are publishing data to the Internet now with increasing frequency — making records public that were once published to books, stored on microfiche, or available only upon specific request in-person.

The greater point is that we're all creating data about ourselves every day — personally, as citizens of our locality, as employees, etc.  Although the "quantified self" movement is relatively young, we're already uncovering introspective ways that we can get smarter about the things that we're doing using data as our guide.

Through collecting and analyzing data appropriately, all stakeholder groups can benefit.  A few quick examples…

• Self — quantifying exercise, meals, etc. for better health. 
• Business — quantifying operations to understand efficiencies in staff, travel, etc.  Mining customer data to understand market segments & purchase intent.
• Public Sector — combing through the data to better isolate fraud potential.  Using data to optimize tax receipts.

The common term across all of them is "efficiency" — using data to become more efficient.  In a hypercompetitive world, efficiency is the way to unlock more profits through decreased costs, increased revenues, and/or improved experiences.

Additionally, data can be predictive and prescriptive.  Let's say you are in a family of four and have bought two cars over the last 8 years.  You have a luxury car with 20,000 miles and an SUV with 200,000 miles.  It's a simplistic case, but the data in this example suggests that you might be in the market for an SUV soon and years away from another luxury car purchase.  That's the predictive part.

Odds are if you got the SUV serviced by a data-driven car dealership, you've heard from them recently about trading in that old clunker in exchange for a shiny, new SUV.  That's the prescriptive part — send everyone likely to buy a new car a new advertisement.  Like it or not, but it happens all the time already… all while many corporate marketing departments are not fully optimized to take advantage of the opportunities available via data.

As we've talked with a number of companies about our work in big, social data, we've discovered a wide range of companies in terms of adoption of data analysis — collection, normalizing, analysis, data-driven decision making.  Some are leading the way, and others still make analog decisions.  To be expected, although we do think the shift towards data is a long-term trend that will be impossible to ignore for most.

In the next blog post, I'll talk more about the mechanics of making sense of it all — from data normalization to analysis and informed decision-making.