Techcrunch ran an article yesterday on the Twitterer who unwittingly, unknowingly live tweeted the raid on Osama bin Laden's compound.  I thought it was a particularly interesting case that underscores a few things about social data mining — an area where we've worked in earnest for about six months and where we are starting to make some real progress.

But first, a quick sidebar:

I've said numerous times that the "social" phenomenon of the present day is really a few things — mapping of the social graph *and* the revolution in real-time/self-published data.

They are inherently different, although all the major social nets (Facebook, Twitter, LinkedIn, and YouTube) do both.

Twitter has mastered real-time data (for now).  As long as they don't freeze developers out of using the data stream, they'll be OK.  If they get too cute, they'll kill the golden goose and developers will innovate around Facebook's status updates and/or a Twitter competitor will emerge.

Facebook is the winner of the social graph (for now).  Similarly, if they just execute their plan and not freeze developers out of the social graph, they'll too be OK.  

So whether or not you like Facebook and Twitter, both are undeniably capturing a record of history like no other entity that ha scome before it.  Historians will have a field day poring over records of Tweets and Facebook updates to recount history as told and experienced by normal people.

Speed and transparency to relevant information are being redefined.  Tech-savvy observers of the bin Laden story knew that the best way to get the latest information was to run a search on http://search.twitter.com — not to listen to the live broadcast of your favorite news outlet.

We discovered that 15 years ago at STRATFOR when we mined Lexis Nexis for relevant data and information as traditional news outlets relied upon tried & true methods of gathering news.  Lexis Nexis was just a digitized collection of articles — certainly higher quality than the public stream of consciousness but more or less conceptually the same.

And even 15 years later, mining the Internet and real-time data is still the name of the game.  Facebook and Twitter users are people who (for their own reasons) talk about things they experience every day.  They do so in the open yet in relative anonymity because of the volume of the stream.  Decision makers who can pull out the intelligent nuggets from amidst the noise will beat other people who rely on others to tell them what is important, and there are implications in a variety of industries.  Consider the following:

  • The aforementionied Twitter user who live tweeted the operation in Abottabad to find & kill Osama bin Laden — that data was in the open hours before any news organization reported it.  News organizations can integrate the wisdom of the crowd into the newsroom like never before.
  • Investors in a particular market or equity can find out about supply disruptions or can mine social networks for information on how employees of a company feel about business prospects.  They similarly can use the global feed or parts of it to monitor sentiment
  • Companies in a competitive market can mine social networks for information on customers, leads, and people with purchase intent.  They can identify specific targets and understand their likes/dislikes.  They can also map influencers of that person, people that person influences, and more or less what makes them tick.

But despite all of the possibilities and all the advances in self-publishing and the Internet, collectively we have not gotten better at getting smarter, faster.  The news organizations, investors, and leading companies of the next decade will be people who learn how to mine real-time data — both what is being said, by whom, and by the additional context that qualifies just how important the information is.