10 Feb, 2016
7 Steps to Consider Before You Kick-start a Big Data Project

"Big Data is the New Oil," "Big Data is Too Big to Ignore." While there could be a debate on who coined these phrases, there is no denying of the fact that ever since the arrival of Internet, Big Data is the most signature buzzword that has caught the imagination of executives like never before, albeit the definition of Big Data continues to evolve. Businesses are increasingly turning to Big Data Analytics Solutions to measure and improve their performance and productivity. Be it banking, retail, insurance, telecom, mining, manufacturing, customer services, and even competitive sports, Big Data is turning out to be a game changer in shaping the future of businesses.

What began as a nascent concept in early 2010, just in a couple of years Big Data gained huge momentum. Historically, organizations have relied on legacy RDBMS systems which were limited in their abilities to process only the structured data. Thanks to the accelerating advancements in information technology and processing capabilities, web usage, social media platforms, and connected devices, an unprecedented amount of semi-structured and unstructured data is getting generated. Simultaneously, the capabilities to store and analyse big data to bring deep insight are also trending up, benefiting all stakeholders.

Here are few more staggering statistics on the amount of big data that is generated across different industries:

  • Facebook gathers and processes more than 10 terabytes of data according to an IBM report
  • An oil rig generates 5 terabytes of data every minute
  • A jet engine generates 10 terabytes of data every 30 minutes

Globally, more than 2.5 quintillion bytes of data is generated every day, i.e., 2.5 followed by 17 zeros!!!

All these example substantiate the same narrative; Data is another resource to economic input, just like labour, capital, and technology.

According to research firm International Data Corp. (IDC), annual spending on Big Data is expected to grow by 23 percent CAGR from 2014-2019 and could reach $49 billion in 2019; a signification proportion of the aggregate corporate spending on IT. Another research study carried out by GE reveals that that 87% of enterprises believe Big Data analytics will redefine the competitive landscape of their industries within the next three years.

So, with the growing conviction that Big Data can bring a paradigm shift in an organization's market positioning and competitive strategy, an inevitable questions companies need to ask is are they ready to switch gears and include a for a Big Data Framework Adoption?".

Big Data Framework adoption can be a daunting undertaking. However, a forward-thinking organization can build a Big Data Ecosystem backed by robust data warehousing and analytics capabilities to position itself ahead of its competition. Findings from Bain & Company showed that early adopters of Big Data Analytics are twice as likely to be a leader in their industries and five times likely to make decisions faster than their competitors.

Having worked with some of the most renowned organizations across industries, Trianz has developed a "Big Data Adoption Framework" which includes seven steps that organizations should consider before they kick-start a Big Data Projects.

1. Understand Industry Point-of-View on Big Data

Gauging prevailing sentiments and acceptance of Big Data potential is the first step an organization can take to begin its transformational journey. Assessing current capabilities and trends as well as appreciating success and failure of near-rivals can help an organization sidesteps mistakes made by others. Additionally, research reports from well-regarded publications like IDC, Forrester’s, and Gartner can assist in benchmarking potential analytics solution providers and vendors.

2. Identify Business Case Proof of Concept (PoC)

Once the potential and strength of Big Data are articulated, it is very important to identify the business cases that could lead to an increase in RoI for the organization. Developing a sound business case with a clear demonstration of associated costs and values and other strategic outcomes can help in getting the executive approval and funding for a pilot phase.

3. Evaluate Current Tools/Technology

After a decision has been made to go ahead with a Big Data pilot phase, the next logical step is to evaluate and finalize the best fit tools and technology stack that meets the current enterprise architectural guidelines. Identify vendors who have good credentials and have demonstrated success in the Big Data field. Once each vendor is scored based on enterprise product/tool evaluation framework, the product with the maximum points can be finalized. It is also critical to select an implementation partner with expertise in executing Big Data projects since such an association can accelerate the project pace with an opportunity for the organization to focus on its data talent.

4. Develop Big Data Implementation Framework/ Process Steps

A thorough evaluation and understanding of the various steps involved in implementing a Big Data Framework can lead to setting up success benchmarks. Over the course of implementation, organizations can use iterative and agile techniques with evolving business requirement to deliver quick solutions for different business units. Therefore, an incremental approach that addresses both short- and long-term aspirations of businesses would be ideal as it provides stakeholders to reassess the budgetary requirements as well as focus on any unanticipated risks.

5. Finalize Architecture for POC/Pilot Project

Organizations which have invested extensively in their Business Intelligence and Information Management systems needs to do their due diligence while finalizing the appropriate architecture. Some elements of the current IT stack might need a redesign to support newer applications. In our experience, most of the organizations favour a modular approach to Big Data with a flexible and scalable functionality, without causing any undue strain on their existing IT infrastructure.

6. Capture Business Measures of Successful POCs

An analysis of pre-implementation expectations with the outcomes achieved from Big Data pilot can help in identifying the success and failure of the effort as well as in developing recommendations to drive future enterprise-wide Big Data initiatives. Realization of 2-3 measurable outcomes can help demonstrate the potential of Big Data Analytics from both an IT as well as business perspective.

7. Envision Big Data Roadmap

Once the business results are proven with PoC, business and IT unit head can envision a long-term Big Data Roadmap for the organization with measurable and meaningful business and financial goals. The execution team can drill down into this roadmap with clearly articulated key metrics, expectations, and values to be gained.