What’s the difference between Business Intelligence and Big Data?

I’m often asked the following question:

What is the difference between Business Intelligence and Big Data?

Before getting into my approach to answering that question, let’s be clear on what we’re talking about. When most people say “Business Intelligence” they’re talking about the class of products that have been implemented in most organizations, not the actual information or knowledge that is derived from the use of these systems. When it comes to Big Data, most people aren’t quite sure what they are talking about…some are talking about the size of the data, some are referring to the approach to analysis and others are talking about the process as a whole.

One of the problems that exists today is ensuring everyone understands what big data is and isn’t. While that is a discussion for another time, in this post I’ll simply say that Big Data isn’t something you buy, implement, configure and start using like you would do with Business Intelligence systems. It’s much more complicated than that.

Back to the original question: What is the difference between BI and Big Data?

I’ve never really been able to answer that question as fully as I’ve wanted to because most people aren’t willing to sit down and listen to me walk through the history of business intelligence and big data and explain their differences.

After many attempts at finding a succinct way to describe the differences, I finally figured out that most people don’t care about the technical differences or the history. Most people just want a ‘sound bite’ answer so I came up with this response:

Business Intelligence helps find answers to questions you know. Big Data helps you find the questions you don’t know you want to ask.

When I answer the question this way, I tend to get the nod of the head and and response similar to “…well that sounds really complicated!”.

It is complicated. That’s the difference between Business Intelligence and Big Data. You don’t have easy, well defined reports and answers with big data like you do with BI. You don’t have a single system to implement and manage with big data like you do with BI.  Don’t get me wrong…BI systems aren’t “simple” and the thought and planning that needs to go into BI systems and planning is very detailed, but BI and Big Data are completely different.

Business Intelligence systems have their place in business. They deliver neat, well-designed answers to neat, well-designed questions. Nothing wrong with that…but most businesses don’t have neat, well-designed questions these days. In fact, most organizations don’t really know what questions they need to be asking.

That’s the difference between BI and Big Data.

Eric Brown

Eric D. Brown, D.Sc.

Eric is a technology and marketing consultant with an interest in using technology and data to solve real-world business problems. In recent years, he has combined sentiment analysis, natural language processing and big data approaches to build innovative systems and strategies to solve interesting problems. EricBrown.com

What does Big Data mean to Database Marketing?

What does “Big Data” mean to Database Marketing?

Over the last year the term “Big Data” has been used progressively and is now part of the discussion in the budget battles taking place in many industries and organizations.  For me, it all started with hearing the word “Hadoop” back in 2010.  Who forgets a word like that?  You won’t.  The second time I heard it, I Googled it and then had a big smile on my face.  “Hadoop” and “Big Data” both mean more value for data, data integration and analytics.  It takes the value of data and data monetization to a new level.  Someday you will thank whoever coined the term “Big Data” and will hear the word Hadoop and not laugh.  There’s nothing worse than sitting in a meeting with IT and being asked “What are your thoughts on Big Data” and having no opinion.   Why should it matter to you- because it is all in the path to a bigger and now validated budget.  It also does not hurt to sound like you know what you are talking about.

“Big Data” budget for everyone

The buzzword in the budget wars right now is “Big Data”.  Everyone wants a piece of the “Big Data” budget – Marketing, HR, Compliance, Finance, and IT.  I also read that there is $200M earmarked for “Big Data” in the U.S. government budget this year.  Now that the word is out, the possibility for the application of “Big Data” is being discovered everywhere.  You could not ask for more exposure around data and funding for data integration and analytics.  For years it was exceedingly hard to justify the expense for data, data integration, data warehouses and analysis tools during budget reviews.  It was often too technical for Marketing Executives to fully appreciate and there was little understanding.

A faster path to direct answers with data

A decade ago, there was little interest in “Customer Intelligence” – and no one had coined the term back then.  We thought customer information was cutting edge but couldn’t find anyone to look at the reports because they were about “customers” not “prospects”.   In the 90’s data integration and matching consisted of homegrown processes that did
variations of exact name, phone number and address match strings, and IT would shut us down when we tried to add more business rules into the matching algorithms.  Employees spent hour’s manually matching records by looking the businesses up on the Internet, which we thought was a revolutionary data matching research tool.   There was the thought that data gave direct answers and there was no concept of what was between having data and getting results.  CDI, data governance, data stewards, and analytics were not part of our vocabulary back then.  We did those jobs but there were no cool words coined yet to describe our technical jobs.  We were in marketing, dealing with marketing executives and an IT organization that also thought we were “marketing” and were not technical like they were.

Feel comfortable saying “Big Data”

Here is a cheat sheet for mastering “Big Data” and feeling comfortable saying Hadoop.  Big Data is all kinds of data.  Examples of types of Big Data: POS and billing transactions, trade, customer interactions via sales, customer service, chat sessions, email, social media, events, and CRM records.  Customer preferences and behavior, customer usage, news, financial reports, hardware, networks and systems, product sensors, system level logs, phone records, employee results and most importantly, third party data.  The list is endless and really, any kind of data will work.

Getting back to Hadoop.  It is one of the many new types of technology that solves the problem of Big Data for businesses.  Think of it as a 3D Data Warehouse, where data does not have to be flat and structured.  Instead of a data refresh and the limitations of storage, data can just be added showing changes in time.  Imagine being able to track a sale made in 2007 to what the customer looks like in 2012 and all of the behavioral data that is in between.  There is finally an application that will give us a multi-dimentional view of data.  In simplest terms, Hadoop[i] is a high-performance distributed data storage and processing system accessible through open source software.  The system stores data on multiple servers and runs parallel processes against the data across servers and then combines the information to provide results.

Initiate the discussion in your next meeting

Still wondering why “Big Data” matters to you?  For database marketing professionals, who has more experience as business users of data, data integration and analytics?  Your experience and skills are now in demand across the enterprise.  The term “Big Data” is relevant right now, sounds impressive, is technical and is new (not really).  It has been around for years to anyone in the data world, but new to those (that approve the budgets).  So, go, grab the words and in your next meeting ask “what does “Big Data” mean to your department, organization or company”.   You could be really impressive and follow it up with “any thoughts on deploying “Hadoop” across the Enterprise”?

[i] Cloudera.  (2011).  Cloudera Ten Common Hadoopable Problems Whitepaper.  Retrieved from http://info.cloudera.com/TenCommonHadoopableProblemsWhitePaper.html

This article is by Rebecca Croucher from inside-data.com.

The Promise of Big Data Analytics and Insights

Cognitive Computing and Big Data are transforming the way businesses, governments, and organizations solve their most complex problems. Data is all around us. But making sense of that data is challenging. It comes from everywhere and in many shapes and formats, structured and unstructured, from the Internet, mobile devices, social media, GPS, RFID, and elsewhere in neat and tidy columns and tables or more random. It is data that is too big and too chaotic to be leveraged by organizations.

The Enterra Cognitive Reasoning Platform™ (CRP) solves these challenges by combining the speed and accuracy of computational computing with the adaptive and predictive capabilities of human reasoning. It automates the analysis of vast amounts of data to uncover previously unknown connections, conclusions, inferences, and deductions that improve decision making across industries or domains.

Transforming Industry

In consumer products and retailing, Cognitive Computing is helping manufacturers and retailers identify precise sensory profiles of consumers to help manufacturers create, market, recommend, and suggest tasty new products to consumers. At the same time, these new technologies are helping companies identify and create the Digital Path to Purchase that drive purchasing by informing, educating, persuading, tracking and recommending goods and services.

In the supply chain, Cognitive Computing and its ability to analyze extraordinary volumes of data is improving operations, reducing waste, enhancing promotions, and ultimately delivering products more efficiently to manufacturers, retailers, and consumers.

In life sciences, Cognitive Computing and Big Data are revolutionizing our understanding of how genetics impact the onset and progression of disease, accelerating the discovering of drugs to treat and cure on the molecular level.

The Enterra CRP is helping some of the world’s leading brands and organizations realize the power of artificial intelligence and use it to transform their operations, supply chains, marketing, and new product development. Within industries and across domains, Enterra is solving some of today’s most complex business challenges to create a world that is better informed by the data around us.


Enterra Solution

A Cognitive Computing Company that specializes in Big Data Analytics and Insights. Enterra Solutions