Business analytics: Dealing with the data deluge

by Urs E. Gattiker on 2013/05/26 · 4 comments 10,514 views

in a analytics taking action

CyTRAP BlogRank: The best data analytics blogs

Amazon, Google, Microsoft, Oracle or SAP?
Yahoo is “…rebooting its languishing photo-sharing site Flickr…” by giving users a terabyte of storage.
Are we storing the right things? Are they all memorable moments?
We outline best practice and show you how to succeed with your analytics.
Business analytics can be described as the methodical exploration of an organization’s data, with emphasis on statistical analysis. This post is the third in a series:

1. Business analytics: Best practices for success (October 2, 2012)
2. Business analytics: Arbitrary rankings, anyone? (November 20, 2012)
3. Business analytics: Dealing with the data deluge (May 26, 2013 – you are here)

Guess what happens in a minute on the web: 204 million emails are sent, 6 million Facebook pages are viewed, and 1.3 million YouTube clips are downloaded (see below).

In addition, 47,000 apps are downloaded, and 20 people have their identities stolen. Another study predicts that by 2015 the number of networked devices on the Earth will be double the number of people.What it is - business intelligence or business analytics - let's not fight about it.

The Smart Data Collective published a nice series on data issues, including:

  1. BI Advice for Midsize Organizations: Keep It Simple
  2. Taming Big Data Is Not a Technology Issue

The series indicates that we may have more data than we can properly handle – technology is not the issue.

Generally, there are three types of business analytics: descriptive, predictive and prescriptive analytics. Descriptive looks at historical data, while predictive tries to determine what might happen, and prescriptive focuses on giving us different options, in which case we choose what we think suits us best, given time and money constraints. The question remains whether we have the right data…

Do data make sense?

Data storage has become dead easy. Flickr recently offered all users 1 terabyte of free storage space in the cloud for their images. Assuming each person’s lifespan is 100 years, this means storing 6250 new pictures each year, or about 17 new images DAILY. Ask yourself, do all those images qualify as so-called Kodak moments? Probably not.

Want to manage the data avalanche better? Do what 5,000 other subscribers have done, sign up for our blog – it is FREE!

CLICK ON IMAGE - Increasing data - we are drowning.

Which data is required for what purpose?

Creating a dashboard is certainly not an easy task, especially when it is supposed to be user-friendly and provide useful information. However, for analytics to work properly it is a must to define what we wish to measure and why. How these data will be used, and for what kind of decision-making will also help determine what must be collected.

To return to my Flickr example, unless we systematically photograph our child’s growth, a few important moments will be missed. But if we shoot anything and everything, we will end up with mostly useless pictures. Therefore, it pays to decide

  • which events must be photographed (e.g., the child’s birthday party – each year),
  • what kind of shots will be kept (i.e. not removed from the storage device), and
  • who gets to curate their publication based on points 1 and 2 (e.g., both parents decide together?).

Accordingly, unless we come up with a cataloguing system (e.g., a main folder for each year with sub-folders), it will be difficult to find our Kodak moments again quickly. For instance, will we have each day, or each month and each day as a sub-folder? Whatever we decide, we have to look at it from the angle of what makes it more useful (i.e. easier to find the image we are looking for) and explain why. The figure below tries to outline how this can be done, including addressing ethical issues and privacy matters.

CLICK ON IMAGE - best to figure out a few key points before collecting data.


We are producing and collecting (e.g., online – in the cloud) increasing amounts of data. Most people and organizations have plenty of pictures, infographics, etc. online for people to see (e.g., Facebook, Flickr, Instagram, Picasa). And let us not forget music files, accounting records

The challenge is to figure out which data can be used most effectively to gain insights about a phenomenon, before making a decision, whether business or personal. I explain some of these data-related challenges for anyone who dislikes statistics here:

FREE ebook: Raise your hand if you hate research. ComMetrics eBook series #2011-01 (783 KB – pdf)

Questions: Please help

Will all these data (i.e. 17 pictures of each day of my life stored on line) get us to make better decisions, will we manage risks better or will we just drown in it?

Remember easy come, easy go?

How do you check if the quality of data used to write a study report justifies the conclusions made by the authors?

What do you see as a real challenge with big data?

Thanks for reading and commenting! I appreciate you sharing your views.

More resources about data bias, Likert scales, parametric statistics, missing values, methodology, etc.

– Lewandowsky, S., Ecker, U., Seifert, C., Schwarz, N., Cook, J. (December 2012). Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest, 13(3), 106-131. Retrieved October 4, from

– see also interview with lead author about article:

Q & A With Psychological Scientist Stephan Lewandowsky (part 1)

Q & A With Psychological Scientist Stephan Lewandowsky (part 2)

– Big data – methodology, research, statistics and parametrics… (just click the URL to query and get some very interesting results – more ComMetrics content).

What is your opinion?

How do you know your data are good enough to let you make the decisions you must? What data-based things do you trust, or not? Any experiences you care to share?

Source: @ComMetrics | Business analytics: Dealing with the data deluge

Urs E. Gattiker, Ph.D. - CyTRAP Labs - ComMetrics.

The author: This post was written by social media marketing and strategy expert Urs E. Gattiker, who also writes about issues that connect social media, strategy, ROI and compliance (click Google Scholar or Microsoft Scholar), and thrives on the challenge of measuring how it all affects your bottom line.

His latest book, Social Media Audit: Measure for Impact, was recently published by Springer Science Publishers; he is currently working on his next book, scheduled to appear before the end of the year.

Connect with ComMetrics on Google+ or the author using: Email | Twitter | | Xing

  • Sandra Fels

    This is a very nice post. You mention:

    “Will all these data (i.e. 17 pictures of each day of my life stored on line) get us to make better decisions, will we manage risks better or will we just drown in it?”

    I would say, no. We will store ever more and be ever less likely to recover it on our PC, in the cloud or wherever these data might be.

    • Urs E. Gattiker

      Dear Sandra
      Thanks for sharing. Absolutely, unless we have things tagged with keywords and we remember, how are we to ever find these things.

      It is already a challenge to find all these data you stored on your PC and now we have numerous places in the cloud…

      I believe the key is really to figure out what we need these data for and then design a system that makes it easy for us to store/archive information we collect under our scheme.

      More likely than not we may have the wrong data to the decisions we need to make :-)

      Sandra thanks for sharing.

  • Urs E. Gattiker

    Big Data Fail

    It cannot be that @AmericanExpress:twitter ads on Facebook show up next to repulsive ads violating women’s rights!

    But this is what can happen on Facebook when the algorithm has a major fail (see image attached). People have complained about this
    See Xing Social Media Monitoring Group:

    Visual example:

    These are more examples that illustrate that big data is fine but unless these are evaluated properly and the conclusion made are correct, a brand’s image can be severely damaged

  • Pingback: Alexa data fail, analysis, business analytics, big data prediction

Previous post:

Next post: