Max Kanaskar is an Associate for Strategy& (formerly Booz & Company). He has extensive experience in IT capability development and data management, and has advised companies in transformational data management programs, including data strategy, management and governance. He’s a regular commentator on big data and technology. His views can be found here.

Follow @DataconomyMedia

Enthusiasm around big data’s potential in creating business value has been driven to a great extent by technology innovations that have been shaping up in the database management and analytic platform market over the past decade or so.  These innovations have been driven by three primary technology related factors:

• Limitations of Existing Data Management Platforms: As Internet entered the business mainstream, new models and paradigms for large scale data management became necessary.  Vertical scaling through addition of raw computing power and disk became increasingly unfavorable as a feasible alternative.  Traditional database and data warehousing platforms ran into cost effectiveness limitations in serving increasingly varying data processing workloads.
• Falling Cost of Technology:  The price of all three workhorses of modern day data management i.e. disk, CPU, and network, continued to fall, enabling new price-performance characteristics.  Use of solid state devices and flash memory is becoming commonplace, more CPU cores mean more processing threads, and Infiniband and other network interconnects allow for speedy movement of massive data sets.
• Computing Platform Innovations: A range of technologies from cloud computing to open source software has enabled the rise of a modern digital platform that is giving rise to new models for sourcing and delivery of data management and analytic capabilities.  Google’s BigQuery service represents just the beginning of this evolution.

These factors have driven the emergence of a plethora of vendors and solution providers in the database management and analytic platform market over the past decade and a half.  Venture capital investors have poured more than a billion dollars in funding data management start up companies. The market today is a hodge-podge of various players: from traditional data warehousing solution providers adding capabilities and/or acquiring data management startups to round out their product capabilities, to new challengers that are rethinking and re-architecting the data management and processing platform from the ground up.  The state of the market today reminds many of the “database wars” of the 1990s when numerous relational database vendors were fighting to capture the database market, ultimately leading to the emergence of a few victors (Oracle, IBM and Microsoft) by the early 2000s.  Similar such developments are bound to shape the database and analytic platform in the coming years.  However, until that happens, the market is bound to be an interesting smorgasbord of products and solutions.

There are four major logical environments (see graphic below) related to data processing and information delivery that these products and solutions ultimately fall in and are competing to support.  These four environments are interconnected by a fifth category of products and solutions: Data Connectors that consists of products and solutions providing integration paths between environments and across legacy and new tools.

In subsequent posts, I will delve more deeply in each of these categories.  For each of these categories, we will look at key solution capabilities, and major vendors and their products and solutions that provide those capabilities.  The categories above are a logical set of categories, i.e. we will find many vendors that offer an integrated single platform that on its own supports multiple environments.  However, we will find it easier to understand this complex and evolving landscape if we attempt to deconstruct the picture logically one piece at a time.

This article was first posted here

Follow @DataconomyMedia

Max is an Associate for Strategy& (formerly Booz & Company). He has extensive experience in IT capability development and data management, and has advised companies in transformational data management programs, including data strategy, management and governance.

Interested in more content like this? Sign up to our newsletter, and you wont miss a thing!

[mc4wp_form]

(Image Credit: Flikr)