The 4 Phases Of Big Data

The combination of smartphones, tablets and connected devices will create a tidal wave of new data for businesses to store and process. Data sources and types are exploding as mobile, the Internet of things and social produce exabytes of structured and unstructured data, commonly known as “big data”. While the concept of managing a torrent of information isn’t new, the challenge of dealing with the three V’s of data management--Volume, Variety, and Velocity --has been taken to a new level by the rise of unstructured data sources, such as social media, mobile application data, video, sensors and other connected devices.

Volume references the amount of content a business must be able to capture, store and access. Variety represents the various types of data that can’t easily be captured and managed in a traditional relational database. For example, a business needs to capture new data sources such as location, motion, and environmental conditions like temperature and humidity. It must also capture images and video in addition to handling more structured data such as forms. Velocity requires analyzing data in near real time. While the existing installed base of business intelligence and data warehouse solutions weren’t engineered to support the three V’s, big data solutions are being developed to address these challenges.

Two weeks ago, IBM released the results of a study it had conducted with the University of Oxford. A full copy of the IBM study can be found here. The study surveyed 1,061 companies from across the globe. The survey found that twenty-eight percent of the firms interviewed were piloting or implementing big data activities. IBM outlined four phases of big data adoption, which include educate, explore, engage and execute. These stages are defined as follows:

• Educate. This phase focuses on knowledge gathering and market observations.
• Explore. After completing the education phase, companies will develop a strategy and roadmap based on business needs and challenges.
• Engage. During the third phase, a business will pilot big data initiatives to validate value and requirements.
• Execute. Companies in the fourth phase have deployed two or more big data initiatives and are continuing to apply advanced analytics.

Of the 1,061 companies interviewed twenty four percent were in the educate phase and another forty-seven percent in the explore phase. Only 6 percent of the respondents had reached the execute phase. The study concluded that big data leadership shifts from IT to business leaders as organizations move through the adoption stages.

In a call with industry analysts, IBM discussed the research findings and provided a list of recommendations for companies. To create value from big data, IBM stated that a company should:

1. Commit initial efforts to customer-centric outcomes
2. Develop an enterprise-wide big data blueprint
3. Start with existing data to achieve near term results
4. Build analytical capabilities based on business priorities
5. Create a business case based on measurable outcomes

We’ve always had data. Social networks and mobile devices simply create more data. Today, we have the opportunity to store and analyze this data more effectively than in the past. Unfortunately, there is no “one size fits all” solution for big data. The solution requirements vary based on criteria such as need for real-time analytics; need to support wide varieties of unstructured data and volume of data. As we can see from the IBM study, big data adoption is in its infancy. This will change as the solutions mature and as companies look for ways to harness business data to create competitive advantage.

What are your big data or mobile challenges? Share them with me on Twitter at @MaribelLopez or Facebook.com/lopezresearch

You can also find a discussion on this topic at #smarteranalytics in Twitter