Business analytics: Dealing with the data deluge

1. Business analytics: Best practices for success (October 2, 2012)
2. Business analytics: Arbitrary rankings, anyone? (November 20, 2012)
3. Business analytics: Dealing with the data deluge (May 26, 2013 – you are here)

Guess what happens in a minute on the web: 204 million emails are sent, 6 million Facebook pages are viewed, and 1.3 million YouTube clips are downloaded (see below).

In addition, 47,000 apps are downloaded, and 20 people have their identities stolen. Another study predicts that by 2015 the number of networked devices on the Earth will be double the number of people.

The Smart Data Collective published a nice series on data issues, including:

1. BI Advice for Midsize Organizations: Keep It Simple
2. Taming Big Data Is Not a Technology Issue

The series indicates that we may have more data than we can properly handle – technology is not the issue.

Generally, there are three types of business analytics: descriptive, predictive and prescriptive analytics. Descriptive looks at historical data, while predictive tries to determine what might happen, and prescriptive focuses on giving us different options, in which case we choose what we think suits us best, given time and money constraints. The question remains whether we have the right data…

Do data make sense?

Data storage has become dead easy. Flickr recently offered all users 1 terabyte of free storage space in the cloud for their images. Assuming each person’s lifespan is 100 years, this means storing 6250 new pictures each year, or about 17 new images DAILY. Ask yourself, do all those images qualify as so-called Kodak moments? Probably not.

Want to manage the data avalanche better? Do what 5,000 other subscribers have done, sign up for our blog – it is FREE!

Which data is required for what purpose?

Creating a dashboard is certainly not an easy task, especially when it is supposed to be user-friendly and provide useful information. However, for analytics to work properly it is a must to define what we wish to measure and why. How these data will be used, and for what kind of decision-making will also help determine what must be collected.

To return to my Flickr example, unless we systematically photograph our child’s growth, a few important moments will be missed. But if we shoot anything and everything, we will end up with mostly useless pictures. Therefore, it pays to decide

• which events must be photographed (e.g., the child’s birthday party – each year),
• what kind of shots will be kept (i.e. not removed from the storage device), and
• who gets to curate their publication based on points 1 and 2 (e.g., both parents decide together?).

Accordingly, unless we come up with a cataloguing system (e.g., a main folder for each year with sub-folders), it will be difficult to find our Kodak moments again quickly. For instance, will we have each day, or each month and each day as a sub-folder? Whatever we decide, we have to look at it from the angle of what makes it more useful (i.e. easier to find the image we are looking for) and explain why. The figure below tries to outline how this can be done, including addressing ethical issues and privacy matters.

Conclusion

We are producing and collecting (e.g., online – in the cloud) increasing amounts of data. Most people and organizations have plenty of pictures, infographics, etc. online for people to see (e.g., Facebook, Flickr, Instagram, Picasa). And let us not forget music files, accounting records…

The challenge is to figure out which data can be used most effectively to gain insights about a phenomenon, before making a decision, whether business or personal. I explain some of these data-related challenges for anyone who dislikes statistics here:

FREE ebook: Raise your hand if you hate research. ComMetrics eBook series #2011-01 (783 KB – pdf)

Questions: Please help

Will all these data (i.e. 17 pictures of each day of my life stored on line) get us to make better decisions, will we manage risks better or will we just drown in it?

Remember easy come, easy go?

How do you check if the quality of data used to write a study report justifies the conclusions made by the authors?

What do you see as a real challenge with big data?

Thanks for reading and commenting! I appreciate you sharing your views.

More resources about data bias, Likert scales, parametric statistics, missing values, methodology, etc.

– Lewandowsky, S., Ecker, U., Seifert, C., Schwarz, N., Cook, J. (December 2012). Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest, 13(3), 106-131. Retrieved October 4, from http://psi.sagepub.com/content/13/3/106.full?ijkey=FNCpLYuivUOHE&keytype=ref&siteid=sppsi

– see also interview with lead author about article:

Q & A With Psychological Scientist Stephan Lewandowsky (part 1)
http://www.psychologicalscience.org/index.php/publications/observer/obsonline/q-a-with-psychological-scientist-stephan-lewandowsky.html

Q & A With Psychological Scientist Stephan Lewandowsky (part 2)
http://www.psychologicalscience.org/index.php/publications/observer/obsonline/q-a-with-psychological-scientist-stephan-lewandowsky-part-2.html

– Big data – methodology, research, statistics and parametrics… (just click the URL to query and get some very interesting results – more ComMetrics content).

What is your opinion?

How do you know your data are good enough to let you make the decisions you must? What data-based things do you trust, or not? Any experiences you care to share?

Source: @ComMetrics | Business analytics: Dealing with the data deluge

The author: This post was written by social media marketing and strategy expert Urs E. Gattiker, who also writes about issues that connect social media, strategy, ROI and compliance (click Google Scholar or Microsoft Scholar), and thrives on the challenge of measuring how it all affects your bottom line.

His latest book, Social Media Audit: Measure for Impact, was recently published by Springer Science Publishers; he is currently working on his next book, scheduled to appear before the end of the year.

Connect with ComMetrics on Google+ or the author using: Email | Twitter | Google+ | Xing

Tagged as: best practice, big data, business ethics, data analytics, data protection, data security, decision-making, Flickr, Google, infographics, key drivers, Koda, making sense, Pinterest, privacy, risk management, trendwatch, Yahoo!