Gartner describes Big Data as being all about three V’s, Velocity, Variety and Volume and lots of each. Big Data is high velocity, high volume data that comes in many different forms from a diverse set of sources in a range of forms and formats (variety). The data tends to be unstructured or semi structured in nature making it difficult to handle through normal relational database structures and methodologies.
Traditionally much of this data has been discarded due to the cost of storage. This is a restriction that is coming to an end, and now that 20gb of Ram now costs less than a 20gb drive did 10 years ago the cost of keeping large data sets in memory is becoming a possibility.
Here’s a blog courtesy of ZDNet that aims to demystify Big Data and look at the trends and developments that are emerging.