Earlier this year I posited that, due to the exponential rate of growth, the amount of data collected for analysis is becoming beyond the scope of the current analytical staffs to examine it all.  And that the answer to growing the cumulative brain power necessary for this exponential growth in analysis is going to have to be machine learning and artifici

The amount of data that is now available for analysis is growing at an exponential rate.  As the hardware to capture the data becomes cheaper, faster and omnipresent, the amount of data collected for analysis is becoming beyond the scope of the current analytical staffs to examine it all. 

The theme of this blog is an examination of forces that would disrupt existing data warehouse implementations.  I categorize these as either long tail or black swan events.

During my first coding class, after the obligatory “Hello, World,” my assignment was to find the least cost path in a matrix of values (linear programming).  The computer took some time to crunch the data to come to a solution.  Now, this is an app on your phone.  

This blog concentrates on the expected unexpected external factors that can have a (negative) impact on your organizations’ Integrated Data Warehouse (IDW).  The current discussions around what NSA can and cannot capture and store for data analysis got me thinking about the biggest elephant in the room:  the government.

A couple of recent articles in Wired got me thinking about just how social media services, and thus the value of the big data that they create, could be under threat from their own customers.

When this blog started, it was based on my 2008 Teradata Partners User Conference presentation on the future data explosion and its impact on the Enterprise Data Warehouse:  “’Long Tails,’ ‘Black Swans,’ and Their Impact on EDW and AEI.”  One could foresee a definite future negative impact on the existing EDW platforms trying to support all this new volume and types of data (social, sensor, etc.).

Most of the “Big Data” examples deal with enterprise analytics being run on terabytes or petabytes of data.  However, this example shows how widespread social data analytics have become.  And, as I live in the wine country, this example is quite close to home.

Darryl McDonald, President of Teradata Applications, tweeted this link today:  a practical example of when many of the subjects discussed on this blog come together. 

Continuing on the “42” theme, Chris Anderson drills done on the impact of big data on health and agriculture. 

“42” is the Answer to the Ultimate Question of Life, the Universe, and Everything.  But for Big Data, the answer is simply “everything.” 

I’ve written a few blogs on the future of 3D printing and its impact on manufacturing and data analytics in the future, or as Chris Anderson stated, “The long tail of things.”  But, up to now, 3D printing has been expensive and mostly for small objects.

Ever wish they would fill-up the pothole quicker?  Municipalities are creating apps to allow the crowd to give them up-to-date notification of problems in the street.  This is in addition to the automated systems that tap into the stream of GPS and cell phone sign

One of the more infamous data usage stories over the last week concerned the removal of the iPhone app, “Girls Around Me.”  This app aggregated Foursquare and Facebook data via their respective APIs to generate a map, with pictures, of wo

Schumpeter’s blog had (yet another) interesting entry a couple of weeks back:  “Now for some good news.”  It reviews a couple of authors’ view that we are on the cusp of future abundance due to upcoming technical breakthroughs.  The four drivers of the future are listed as the:

“All Things Digital” from the WSJ site pointed to the latest sales estimates for gadgets over the holiday period for 2011.  What stood out was the huge decline in hand-held gadgets:  specifically, dedicated gadgets for which “there is a

I will admit it:  I did not “get” Twitter.  When it first came out, I could understand news organizations and media tweeting information, and well-known personalities tweeting the minutiae of their daily lives.  But I couldn’t understand the draw of the remaining 99% to also publish their lives on a real-time basis.  But my curmudgeonly world filter needs

Over the last couple of years, I have discussed on this blog a number of data explosions coming down the line that will have a huge impact on data warehouse implementations now and into the future.  Many of these topics now fall under the umbrella of “big data.”

Schumpeter wrote a column in last week’s Economist that did an excellent job in summarizing many of the big data sources that I have addressed on this blog.  It is based on a McKinsey Global Institute write-up

Mobile data and use is driving major changes to many industries, and the use of mobiles – especially smartphones – will be a growing force over the next decade across the globe.

3D printing hit the cover of The Economist last month.  It is now “officially” having a major impact on business and, as of result, on the required analytics and underlying data warehouse structures to support the analytics

I finally caught up on my reading over the holidays – funny what 12 hours of captivity in coach will do – and I highly recommend the November 6, 2010 special report from The Economist on Smart Systems. 

Nice article today in the NY Times on 3-D Printing, one of the enablers of the Long Tail concept for manufacturing.

The Wall Street Journal is running a series, “What They Know,” this week on Web privacy.

What caught my eye was today’s article on Web analytics, “On the Web's Cutting Edge, Anonymity in Name Only.”  A quote from the article, “firms like [x+1] tap into vast databases of people's online behavior—mainly gathered surreptitiously by tracking technologies that have become ubiquitous on websites across the Internet. They don't have people's names, but cross-reference that data with records of home ownership, family income, marital status and favorite restaurants, among other things. Then, using statistical analysis, they start to make assumptions about the proclivities of individual Web surfers.”

This blog was started to discuss Chris Anderson’s Long Tail concepts and how they relate to Enterprise Data Warehouse (EDW) implementations. Well… Mr.

As logistics become less expensive, more instantaneous, and support a greater variety of goods, the long tail category of "candidate products" continues to expand. For example, 3-D printers.

I’m going to catch up on some of my reading from over the summer. The common theme in this entry on the unabated explosion of data, even when it seems that some industries are scaling back.

First, we have a couple of articles from The Economist which IMHO is the world news magazine for nerds :-). From last June’s technology quarterly, we have two articles: "Sensors and Sensitivity" and "The Connected Car".

One of the first areas I explored when considering the long tail’s impact on EDW was the growth in data storage requirements. Supporting the long tail impacts the overall EDW storage requirements in two ways: width (expansion of the data model) and depth (more history). Let us look at one common subject area for all enterprises: Party (customer).

In this blog we ask the question, “How do the Long Tail concept and Black Swan events relate to and impact the Enterprise Data Warehouse and Active Enterprise Intelligence?” This blog will be a discussion platform for the Data Warehouse disruptions caused by these concepts and how to accommodate them.