Why 'Big Data' is not just about volume

Big data is a popular term used to describe the increasingly data-rich landscape of tomorrow, but it diverts attention away from equally challenging aspects, other than volume.

Too narrow a focus will force massive reinvestment in two to three years to address the other dimensions of big data.

Big data poses a significant dilemma for CIOs and business leaders because the investment required in emergent technology and the rapidly evolving discovery of the technical issues represents a significant risk.

Worldwide information volume is growing annually at a minimum rate of 59 percent annually, and while volume is a significant challenge from a storage and analysis point of view, business and IT leaders really need to focus on other dimensions of information, including:

• Variety: IT leaders have always had an issue translating large volumes of transactional information into decisions. Now there are more types of information to analyse — mainly coming from social media and mobile (context-aware). Variety includes tabular data (databases), hierarchical data, documents, e-mail, metering data, video, still images, audio, stock ticker data, financial transactions and more.

• Velocity: This involves streams of data, structured record creation and availability for access and delivery. Velocity means both how fast data is being produced and how fast the data must be processed to meet demand.

• Complexity: Different standards, domain rules and even storage formats can exist with each asset type. For example, an information management system for media cannot have only one video solution.

While big data is a significant issue, the real issue is making sense of big data and finding patterns in it that help organisations make better business decisions.

Like this article? Sign up for our weekly CIO Strategies bulletin by clicking here.

Gartner advises that big data will represent a hugely disruptive force during the next five years — enabling levels of insight into customer demands that are currently unachievable through any other means.

Additionally, the ability to mine data at this scale will provide a much greater understanding of the underlying physical, societal, financial and business environment.

Organisations that successfully master the technical challenges will find themselves better equipped with more accurate data about the present, able to create more accurate forecasts about the future and, for the leaders, an almost prescient ability to spot subtle changes in dynamic and volatile markets and make the right choices in order to be able to benefit from their new knowledge.

New technologies and applications are emerging (examples include Hadoop and MapReduce) and should be investigated to understand their potential value.

Most importantly, IT leaders must not become focused on the obvious issue of volume alone.

CIOs, in particular, must consider devolving their day-to-day oversight over routine IT infrastructure issues in order to devote time to drive their organisations' evolution toward big data.

Stephen Prentice is a vice president and Gartner Fellow in the Executive Leadership and Innovation Group.