I will admit it:  I did not “get” Twitter.  When it first came out, I could understand news organizations and media tweeting information, and well-known personalities tweeting the minutiae of their daily lives.  But I couldn’t understand the draw of the remaining 99% to also publish their lives on a real-time basis.  But my curmudgeonly world filter needs to adapt as the world of data analytics has seized the opportunity of these new data flows.  Some of the articles that I noted lately:

Decoding Our Chatter” references real-time tracking of events – and by real-time, the first example is that the tweets from this year’s Virginia earthquake arrived in New York before the shockwaves.  The WSJ article focuses on the financial investing opportunities and political trend tracking, but access to social data can also be used to adjust inventories, supply chains, routing, personal, etc., in response to large and not-so-large events, and then on to product popularity trends.

For the latter point, the article “Can Twitter Predict the Future?” delves into using Twitter data to predict trends.  In the article, pairing the raw data available with analytics has been shown to help with sales forecasts for video games and music.  Certainly this can be expanded to most product categories, especially those that are trend or fashion driven.  And trends can apply to practically any product category or service, whether it is the popularity of particular type of wrench, the latest kitchen appliance or a consulting skill.  “Sophisticated methods based on natural-language analysis of tweets, blogs, or Facebook pages, by contrast, hold greater disruptive potential. As users of social media grow accustomed to sharing highly personal information, apparently unfazed by market-research outfits like WiseWindow watching their every step, the feelings and intentions of hundreds of millions of people are there for data-hungry computers to see.”  For “data-hungry computers” read “EDW.”

Given the promise of social data analytics, there is still the issue of implementation.  “Sipping from the Fire Hose” references over 230M tweets a day – not a data source for the faint hearted.  Although the data analytic services referenced in the article offer social media consolidation and filtering services, your organization still needs to bring a very, very large volume of relevant data in-house to perform the proprietary analytics that pay the bills. 

When I initially discussed the implications for the use of social data in the EDW, I was only thinking of using an RDBMS for the analytics.  There are tools that have been available for a number of years that will format the non-relational social data for RDBMS use.  But the trend now seems to be to use a separate DBMS for the initial unformatted data analytics – “big data” – before extracting the results for use by the EDW.  This follows the rule of using the right tool for the right job, and avoids filtering out any nuances of the social data before executing the initial analytics to identify the trends that will give your organization the greatest competitive advantage. 

Either way, social data analytics has now become another tool that has gone from “blue sky” to an everyday requirement to compete in the advanced war of data analytics.