<a href="http://egroup.jasonmangone.com/wp-content/uploads/2012/11/big-data.jpg"><img class="alignnone size-medium wp-image-7695" title="big data" src="http://egroup.jasonmangone.com/wp-content/uploads/2012/11/big-data-300x147.jpg" alt="big data" width="300" height="147" /></a>

The following article was featured in our most recent eGroup Advantage Newsletter. It's a big data conversation with our Data Services Practice Lead, Ian Jones. Give it a read to understand how eGroup is demystifying big data for our customers.
<h1><strong>Demystifying Big Data</strong></h1>
<strong><em>eGroup has launched a new practice dedicated to helping customers gain business insight and competitive advantage through data analytics.</em></strong>

“Big data” is the one of the biggest buzzwords in the IT industry today. Like most buzzwords, it is somewhat vague and frequently misused. Is it simply a large amount of data? If so, how much? And what are you supposed to do with it? Arcane terms such as “analytics,” and strange-sounding technologies such as Hadoop, do little to clarify the matter.

“A lot of organizations are wondering what ‘big data’ is and how it might benefit them,” said Ian Jones, Data Services Practice Leader, eGroup. “It really boils down to taking all the data a business generates and using it to make better business decisions. We’re engaging with some of our larger customers, particularly in healthcare, from a big data standpoint. But the concepts and methodologies can benefit organizations of all sizes and in a wide range of industries.”

eGroup has launched a data analytics practice to help customers take advantage of big data tools and techniques to improve decision-making and gain new business insight. eGroup can cut through the hype and help customers understand how data analytics might fit within their organization. And because of its engineering skills and industry relationships, eGroup is uniquely positioned to deliver the technologies needed for effective data analytics and business intelligence (BI).

“Our vision is to demystify big data and business intelligence for customers,” Jones said. “Our engineering team can help customers ask the right questions of their data and develop strategies for identifying, analyzing and incorporating that data back into their business.”

<strong>Looking for Opportunities</strong>

The term “big data” is used to describe the massive amount of data generated by today’s line-of-business applications that cannot be analyzed efficiently using traditional analytical tools. Although the definition is necessarily subjective, most analysts use the term in reference to petabytes, exabytes or even zettabytes of data.

Of course, no organization stores all its data in one gigantic data warehouse. On the contrary, information may be locked in a variety of different applications and stored as unstructured files. So the first step is to aggregate the information to be analyzed.

“One of the key components of being successful with big data is breaking down data silos,” said Jones. “Organizations have many different applications with different datasets that result in silos with vertical communication. Healthcare is the perfect example. Healthcare organizations have financial systems, electronic medical records, data coming out of emergency rooms and many other data points. Big data analytics seeks to glean intelligence from all of this information, so the data has to be extracted from those sources and combined into one pool which allows for horizontal communication across those different systems.”

Once the data is aggregated, informatics and data science come into play. These disciplines use software and mathematics to discover patterns in the data that humans cannot perceive.

“In the healthcare field, the utopia of big data is predictive healthcare — the theoretical point when we’ll be able to take all the data generated in a hospital, analyze it in real time and provide feedback to clinicians and physicians to aid in diagnoses as well as how to effectively treat that patient. That’s something a lot of hospitals are working toward,” Jones said.

“But healthcare is merely an example. The whole point of big data is enabling companies to uncover opportunities they didn’t know they had and respond to them at a speed that meets the needs of their business. Organizations are generating a tremendous amount of valuable data, and every industry has an opportunity to use that data to improve their business.”

<strong>Not Your Father’s BI</strong>

BI technologies have been around for years — as long as there have been computers, really. But datasets are getting so large that traditional BI systems cannot process them in real time. Big data technologies provide the raw processing power needed to perform these analyses.

“When selecting a big data platform, you have to consider not only the size of the dataset but the speed with which the business needs to process and ingest it. Products like Hadoop and Greenplum provide a distributed, parallel processing platform and clustered computing to allow faster, real-time data analytics,” said Jones.

Another key feature of big data tools is the ability to process unstructured data such as text files and web pages. Hadoop in particular was designed to extract and analyze information that does not fit neatly into traditional database schemas.

“Unstructured data is where the real opportunity lies with big data,” Jones said. “And along with unstructured data there’s metadata — data about the data. Whereas business intelligence is about transactions, big data is about interactions between various sources of information. Context becomes very relevant.”

But Jones stresses that it’s important to not get hung up on the “big data” buzzword.

“We’ve taken a step back and said let’s stop talking about big data and just talk about data,” he said. “Big data is a very specific subset of the problem of how we ingest, analyze and process data to make business decisions. That’s really the core of it. Some or our customers need the tools and systems associated with big data and some don’t. The point is to use the data they have to drive their business, whether with big data analytics or more traditional business intelligence tools.”

<strong>Challenges and Opportunities</strong>

Technology is only one aspect of the big data paradigm. Organizations will face obstacles with finding the right talent to implement their big data strategies. They will also have to be prepared to deal with the security and regulatory issues that may arise.

“Financial and healthcare systems, for example, have a lot of controls in place to prevent security and regulatory issues,” said Jones. “As you’re pulling data from all these different systems and putting it into a consolidated system, you wind up with a big dataset that could include information you have to segment and not allow certain people to see.

“But probably the biggest issue is the cultural shift that needs to occur. It’s important to have deep organizational integration when you’re talking about big data because it affects everyone. Everyone has to participate in it.”

Industry experts say organizations that jump on the big data bandwagon now will be well ahead of the competition in terms of operational efficiencies. Some experts have suggested that big data analytics ultimately will become essential to driving innovation and optimizing business performance.

“Right now, big data is an opportunity,” Jones said. “But as competitors latch onto it, it can become a competitive threat. Our goal is to help customers understand that big data is nothing magical. There’s a lot of opportunity through traditional analytical and integration tools to do a better job of asking the right questions of the data customers already have. That’s the number one takeaway.”