Crowd-wisdom fails businesses

by Urs E. Gattiker on 2009/12/10 · 38 comments 23,546 views

in e marketing 101 serving a need

In his book The Wisdom of Crowds (2004), James Surowiecki popularized the notion that, under the right conditions, canvassing the aggregate opinions of many people could be more efficient than relying on the expertise of a few.

Image - Jeff Howe applied this approach to decision-making using the buzzword ‘crowdsourcing’ in a Wired article in October 2006 (see video below).

Crowdsourcing assumes that customers know best what they want and need. Hence, more heads are better than one. We discuss why crowdsourcing may fail in a few important situations that concern social media.

Article source: Crowd-wisdom fails businesses

Crowds innovate – NOT
For us at ComMetrics, innovation is a step-by-step process (idea to prototype) where each stage of development is combined with regular measurements of factors critical to achieving success. For example, time used and money spent leading to success in the market, as reflected by new subscribers and their feedback.

The ideas came from various places but then we went to the lab and built. Social networks came in when we had the prototype and wanted feedback.

Our approach is reflected by Dyer, Gregersen and Christensen’s The Innovator’s DNA (December 2009). Five ‘discovery skills’ separate true innovators from the rest of us (Harvard Business Review). The authors concluded that innovators apply these behaviors more skillfully than the rest of us.

    Associating: The ability to connect seemingly unrelated questions, problems or ideas from different fields.
    Questioning: Innovators constantly ask questions that challenge the common wisdom – why, why not and what if?
    Observing: Discovery-driven executives scrutinize common phenomena, particularly the behavior of potential customers.
    Experimenting: Innovative entrepreneurs actively try out new ideas by creating prototypes and launching pilots.
    Networking: Innovators go out of their way to meet people with different ideas and perspectives.

It seems a bit naive to think that going to Dodger Stadium or the LA Coliseum in the hope that most people attending the game might be exhibiting the above behaviors, and therefore help us innovate faster…

Crowd-wisdom helps consumers – NOT necessarily
While crowds may not innovate, they still provide wisdom when it comes to product reviews. Superusers’ product reviews on Amazon or eBay influence many. One could ask how reliable these ratings and reviews are. A recent comment on a blog post addresses this in more detail:

    Dear Bridget and Lin,
    This is a really interesting post. Some would suggest that explaining why the current fashion was going to end in our financial disaster would have been dangerous, especially if it then turned out that the crowd was right.
    => Wall Street Blues: 4 lessons learned

    A real concern is the wisdom of crowds who are herded by power-users writing the first review for a product. Any attempt to turn mob opinion into a test for truth is pernicious.

    The notion that a book might be a must-read because it is highly ranked by many on Amazon does not make it Nobel prize material. The earth did not stand still just because Galileo fell out of favor, nor has evolution been shown to be false due to the faith of believers.

    Hence, product reviews driven by superusers and crowds who follow just means that the wisdom of crowds can only be conventional. Volume against quality.
    => SocioTwitting: Developing metrics for Twitter – volume vs. influence

    Thanks for sharing this,
    Urs – @ComMetrics

Thumbs Up or Down works but fails to explain why
Crowds do not drive and bring innovation to successful fruition in the form of a marketable product. Nor are they the best source for assessing quality – the one that shouts the loudest is heard the most.

Nevertheless, crowds can tell you if they like or dislike something.  For instance, Bonobos found that they can come up with a name and choose the one they like the most. Bonobos emailed customers and asked them to name a new pair of trousers – the winner was the Dark and Stormys brand.

But using crowds for things like A/B tests (i.e. comparing several groups’ reactions – including a control group – to different versions of a webpage to improve it) or getting the thumbs up or down sign risks two things:

    a) crowds voting with their fingers still leave you with the question why, and
    b) small changes (e.g., darker font for titles to make for easier reading when viewing a screen in a room with bright lighting) can result in product A being preferred over B again.

To minimize the chance of throwing the baby out with the bathwater, it is best to talk to some clients to find out the why, as we had to learn the hard way when re-designing this blog.

More resources about crowdsourcing and innovation

YouTube Preview Image

Bottom line
Crowdsourcing depends on two things:

    1. a very specific call to action, AND
    2. the clear understanding of participants’ motivations.

These two conditions for successful crowdsourcing are illustrated with the UK Guardian’s call for readers’ help digging through MPs’ expense-documentation to identify individual claims and what merits further investigation. It specifies what members are asked to do and the motivation: to catch one’s local MP if they cheated taxpayers.

Take-aways
There are some crucial things to remember about innovation when crowdsourcing enters the pictures.

    1. Real innovators do not swim with the school: Fashion changes may be a mystery, but forming an independent opinion and pursuing the idea is what characterizes successful innovators – not crowds.
    2. Crowdsourcing may border on exploitation: Lowering a company’s overhead by using a web-based workforce with minimal rewards for non-winning participants (e.g., Jeremiah Owyang crowdsourcing his blog’s banner design means winner takes it all while Jeremiah pays for the one design he likes – exploitation).

Please, leave a comment! We love to hear your thoughts: how do you think crowdsourcing can work with Web 2.0 applications or social media monitoring? Here is a chance for anyone with first-hand knowledge (this means you!) to share your insights.

Tagged as: achieving success, Bonobos.com, buzz marketing, crowdsourcing, innovation, measurement, social media measurement, stage of development, wisdom of crowds

  • Pingback: Urs E. Gattiker

  • Pingback: Ryan Opaz

  • Pingback: Rudi Shumpert

  • Pingback: Urs E. Gattiker

  • Pingback: World Economic Forum

  • http://twitter.com/RRS_ATL Rudi Shumpert

    Interesting read. I really liked the first take-away “Real innovators do not swim with the school”.

    However, I think that those innovators do watch and observe the school. I think that unless you have at a minimum, a basic understanding of the needs or wants of the crowd, how can you innovate or create something that will be used and successful. One of the better books I've read on this lately is “Ground Swell”.

    -Rudi

  • Pingback: Trevor Rotzien

  • http://twitter.com/soenke_d Soenke Dohrn

    I think as opposed to Jeff Howe, James Surowiecki's point was utilising the wisdom of crowds by unlocking it under special conditions. These are:
    a) participants must not be personally aware of each other's participation
    b) all participants are fed with the same quality and quantity of information

    Just take the example James has given regarding the search for the lost submarine vessel. There was no opinion leader and domain experts participating in the rescue mission didn't know each other. Yet each were given the same dossier containing identical information. By piecing together bits and pieces of possible solution strategy, the research mission commander was able to devise a possible location for the sunken vessel not one single individual was pointing to. Yet, the location that emerged out of the data was precisely the spot of the sunken vessel.

    Also Andrew McAfee points out in his book Enterprise 2.0 (p 126) that prediction markets can be used by businesses to gain access to information of knowledge holders their knowledge workers usually would never share a network tie with. Quote “Predicition markets are an ESSP [enterprise social software plattform] that allows untied people to interact.” In all types of markets information is generated via prices and it is this principle which prediction markets are counting on.

    So while James Surowiecki pointed out a knowledge repository and method, Andrew McAfee built on that method and offered a process interface for gaining access. Thus the point to remember is that for reaping the wisdom of crowds there is a specific technique available which only works under certain conditions.

    According to Jaberg, 2005 an invention only becomes an innovation through its economic success. And even this definition from the scientific school holds little value to the actual businesses out there. We have it in our research project that SME think of themselves as innovative when they redesign their administrative processes. So from the scientific point of view I have to strongly agree that a crowd as such can never be innovative but instead can only offer ideas. It is the job of the knowledge worker to find ways to transform the idea into an economic success and become retroactively an “innovator”. Hence I disagree with the title of this post. It is not the crowd-wisdom failing business but rather applying ineffective methods by knowledge workers. It seems a bit like offering the wrong key for a treasure chest and fighting off complaints by saying the chest lock must be broken.

    Kind regards,
    Soenke

    • http://My.ComMetrics.com Urs E. Gattiker

      Rudi,

      Thanks for the point above and yes I have to agree innovators watch the school and then still try to find their unique approaches to make it happen – taking the idea, building a prototype and getting the product to the market.

      Soenke
      Nice comment really nice. And yes we agree that for “reaping wisdom of crowds there is a specific technique available which only works under certain contiditions.”
      I do disagree however that I may have implied that because our key fails to open the treasure chest the latter must be broken.
      What I tried to convey was that how we have tried to apply wisdom of the crowd is

      a) in some cases not meeting the conditions as both Surowiecki and Howe outlined at least in some cases as we try to apply the concept in business AND
      b) meeting one but not both of the conditions (clearly defined challenge/task and motivation of participants is known).

      Your example of the vessel is a good one. It is a clearly defined objective and motivations of participants are known and shared, find the sunken vessel.

      What matters is how we apply ideas and in this case your comment points out that unless we do it in a certain way we may be unable to reap the rewards. So true.

      Thanks so much for sharing and I am looking forward to you both, Rudi and Soenke to share your insights with me and the readers again on a future post.

      Good night and merci.
      Urs

  • Pingback: Rick Mans

  • Pingback: Peter EG

  • Pingback: Peter Olins

  • http://www.ppbdlaw.com/ Brian Pangrle

    I found crowdsourcing to be quite enlightening – everyone should try it. Also, it's probably worth considering a distinction between crowdsourcing and open innovation (see, e.g., P&G and Chris Thoen).

    I believe that “crowdsourcing” is too broad of a concept/experience to be limited to merely a search for opinions or facts. It's a dynamic process that exists in a context (time, motivation to respond, geography, impact of other local or world events).

    In a “search for facts” context, crowdsourcing certainly requires carefully crafted questions and careful analysis of responses for possible underlying facts. If such an analysis leads to commonalities in underlying facts – then that begins to sound like a basis for further investigation.

    More broadly, I believe that “necessity is the mother of invention” and that humans are inherently creative. From my experience as an inventor and intellectual property attorney (dealing with both artists and inventors), desire, imagination, a need to solve a problem, express feelings, place observations in a common context, etc., lead to the most fascinating creations (whether merely artistic, utilitarian or a combination of both).

    While working with artists in New Mexico (including indigenous artists), when questioned, most of these artists found creation to be a human/spiritual experience (usually with a gesture such as finger pointing to heart or hand over the heart) – often, it was viewed as a natural state of being. I've found the best, serial inventors to answer similarly – but using slightly different words and gestures. Serial inventors and artists are prolific fountains of creativity. Further, they release their works to make room for more. The inventor that thinks “this is my baby” (and usually his or her only child) is likely to have 100% of nothing instead of a few percent of something grand.

    With that said, I believe crowdsourcing can spark creation. I find that reading responses to crowdsourced questions teaches me about humans – and how we think, what we desire, etc. It also teaches me about my “questions” and preconceived notions. While some of these preconceived notions may be proven true and some proven false – the completely unexpected responses, are for me, golden. In this sense, crowdsourcing can inspire creation, whether its artistic, inventive and innovative.

    Crowdsourcing is very worthwhile – when it's used as a basis to better understand and serve humanity (and the world in general), the term “exploitation” misplaced.

    Brian

    • http://My.ComMetrics.com Urs E. Gattiker

      Brian,

      Great comment, hope @Chris_McGeehan will add his about crowdsourcing as well.

      I found your statement,

      – “… necessity is the mother of invention” and that humans are inherently creative.” and

      – “… Crowdsourcing is very worthwhile – when it's used as a basis to better understand and serve humanity (and the world in general), the term “exploitation” misplaced.

      I agree that an invention has to solve a problem to make it the market place… and crowdsourcing is surely worthwhile. As I pointed out in my post, this applies if it is:

      1. a very specific call to action, AND
      2. the clear understanding of participants’ motivations.

      So we agree on this. However, if we crowdsource programming or design tasks as an example, we then choose the one we like the most. In turn, all the others that submitted receive nothing.

      This can go so far that companies do this for many jobs, a cheap way to get great ideas, designs and whatever. If we push it too hard, those participating take all the risks while those giving out the tasks to crowds to solve the problem take little if any risks.

      For me this is a dangerous situation and for lack of a better word I used “exploitation.”

      Participating in these 'contests' means that many self-employed participants cannot survive with the few jobs/contracts they might get out of it.

      Hence, crowdsourcing can be beneficial for all parties but we have to put in place checks and balances.

      Brian, thanks for your effort and time taken to share your insights. Look forward to your next comment.

      • http://www.ppbdlaw.com/ Brian Pangrle

        Agreed, we should all strive to maintain fairness and avoid exploitation. For one detailed perspective, I'd recommend Harvard Law School Prof. Jonathan Zittrain's http://futureoftheinternet.org/. He's commented on Amazon's Mechanical Turk (a system that pays workers). A recent lecture at Stanford Law School was entitled “Minds for Sale”: http://www.youtube.com/watch?v=Dw3h-rae3uo. He calls it “cloud labor”.

        Based on my experiences in some of the world's impoverished communities, I believe Mechanical Turk has potential to do good and indeed very well may be doing good. That said, you as well as Prof. Zittrain raise important points that should be given serious consideration. Thanks for providing a forum for this dialog.

        Brian

        • http://My.ComMetrics.com Urs E. Gattiker

          Brian

          Thanks for the feedback, I find both URLs (one text the other video) very helpful.

          Thanks for sharing.

          Look forward to your next comment made on a future post. Merci.

  • Pingback: LOiKLiOT

  • http://www.catseyemarketingblog.com/ Judy Dunn

    Urs,

    Hope I didn't lose too much by not following your links (will certainly do that when I can set aside some time), but just wanted to say that this article is top-notch, full of interesting and intriguing ideas.

    What I really like is that you spent little time here on what most people (creative professionals, at least) focus on when the topic of crowdsourcing comes up: the unfairness of using the ideas of the masses to get to that one “best” idea or product.

    I see the crowdsourcing mentality at work at many levels in our society, and particularly in the business world. Book reviews are one thing. (Although Amazon ratings can certainly help authors sell their books.) But, perhaps more disturbing, are the ways the awards for innovative businesses and best practices are being determined—by vote. So, as you say, it has nothing to do with quality, but rather “the one who shouts loudest is heard.” I smile when I see people on Twitter asking their followers to vote for their business for the “best of this or that.”

    I also think that crowdsourcing on the level of the masses can turn into mob mentality, with people voting for what appears to be the most popular choice.

    I love the Five Discovery Skills. Need to pick up The Innovator's DNA. It takes a certain level of intelligence (and hard work) to practice those skills. I can see how crowdsourcing might seem like an inviting alternative.

    What I see on Twitter is a kind of mini-crowdsourcing, where an idea is tossed out, perhaps a possible topic for a blog post, and diverse opinions are collected that help an author tackle a new subject from all angles. I like that kind of crowdsourcing. :-)

    Thanks for the thought-provoking article. Love to start my day this way!

    Judy

    • http://My.ComMetrics.com Urs E. Gattiker

      What can I say, @CatsEyeWriter you just made my day…. actually it is getting to be evening over here in Europe and I just read your comments now.

      I agree with your idea about Twitter. One can share an idea and get feedback from followers or learn in many other ways from people who use micro-blogging as well.

      While it can help, nevertheless, the creative nurturing still comes from one individual and not the crowd I believe.

      Mini-crowdsourcing – have not come across this term yet but I love it :-)

  • http://My.ComMetrics.com Urs E. Gattiker

    What can I say, @CatsEyeWriter you just made my day…. actually it is getting to be evening over here in Europe and I just read your comments now.

    I agree with your idea about Twitter. One can share an idea and get feedback from followers or learn in many other ways from people who use micro-blogging as well.

    While it can help, nevertheless, the creative nurturing still comes from one individual and not the crowd I believe.

    Mini-crowdsourcing – have not come across this term yet but I love it :-)

  • oscar

    My personal view is that crowdsourcing is essential in combination with other resources. Crowdsourcing itself does not create innovation or new products, but it can lead to new innovations. Crowdsourcing is great for products that are already in the pipeline; the idea and concept is already developed. Innovation is driven by people who are capable to interpret the information and associate this feedback from the crowd to new products/services (The Innovator's DNA).

  • http://My.ComMetrics.com Urs E. Gattiker

    Oscar , I agree, the line b/w primary and derivative innovation is not always clear cut.

    I have tried to address this in the innovation cube shown here: http://cytrap.eu/files/ComMetrics/2009/image/12…

    Primary research is what leads to primary innovation and is based on work evolving out of the testing of theoretical relationships. Hence, the invention of plastic was a primary innovation, while its use in cars or households led to derivative innovations.

    Primary innovations will eventually ‘trickle down’ to reach applied research, which is responsible for the creation of derivative innovations – everything from plastic mop handles to microcomputers and the iPhone cover options.

    I tried to visualize this with the above cube published in 1990. But I am still struggling with the concept myself.

    Oscar, I agree with you that the marketplace is the most important factor in determining the kind of derivative innovations that applied research will create.

    In fact, if there is market for certain types of plastic such as freezer or garbage bags, applied research would never have made them as common as they now are.

    But all this suggests that crowds are rarely if ever the root of such smart and innovative uses of derivative technology such as Twitter or SMS messaging to mention two examples. But for fine-tuning as part of a focus group crowds can be invaluable.

    Thanks again for posing these intriguing questions and challenges. I look forward to your next comment.

  • http://My.ComMetrics.com Urs E. Gattiker

    Oscar , I agree, the line b/w primary and derivative innovation is not always clear cut.

    I have tried to address this in the innovation cube shown here: http://cytrap.eu/files/ComMetrics/2009/image/12…

    Primary research is what leads to primary innovation and is based on work evolving out of the testing of theoretical relationships. Hence, the invention of plastic was a primary innovation, while its use in cars or households led to derivative innovations.

    Primary innovations will eventually ‘trickle down’ to reach applied research, which is responsible for the creation of derivative innovations – everything from plastic mop handles to microcomputers and the iPhone cover options.

    I tried to visualize this with the above cube published in 1990. But I am still struggling with the concept myself.

    Oscar, I agree with you that the marketplace is the most important factor in determining the kind of derivative innovations that applied research will create.

    In fact, if there is market for certain types of plastic such as freezer or garbage bags, applied research would never have made them as common as they now are.

    But all this suggests that crowds are rarely if ever the root of such smart and innovative uses of derivative technology such as Twitter or SMS messaging to mention two examples. But for fine-tuning as part of a focus group crowds can be invaluable.

    Thanks again for posing these intriguing questions and challenges. I look forward to your next comment.

  • http://My.ComMetrics.com Urs E. Gattiker

    Oscar , I agree, the line b/w primary and derivative innovation is not always clear cut.

    I have tried to address this in the innovation cube shown here: http://cytrap.eu/files/ComMetrics/2009/image/12…

    Primary research is what leads to primary innovation and is based on work evolving out of the testing of theoretical relationships. Hence, the invention of plastic was a primary innovation, while its use in cars or households led to derivative innovations.

    Primary innovations will eventually ‘trickle down’ to reach applied research, which is responsible for the creation of derivative innovations – everything from plastic mop handles to microcomputers and the iPhone cover options.

    I tried to visualize this with the above cube published in 1990. But I am still struggling with the concept myself.

    Oscar, I agree with you that the marketplace is the most important factor in determining the kind of derivative innovations that applied research will create.

    In fact, if there is market for certain types of plastic such as freezer or garbage bags, applied research would never have made them as common as they now are.

    But all this suggests that crowds are rarely if ever the root of such smart and innovative uses of derivative technology such as Twitter or SMS messaging to mention two examples. But for fine-tuning as part of a focus group crowds can be invaluable.

    Thanks again for posing these intriguing questions and challenges. I look forward to your next comment.

  • http://My.ComMetrics.com Urs E. Gattiker

    Oscar , I agree, the line b/w primary and derivative innovation is not always clear cut.

    I have tried to address this in the innovation cube shown here: http://cytrap.eu/files/ComMetrics/2009/image/12…

    Primary research is what leads to primary innovation and is based on work evolving out of the testing of theoretical relationships. Hence, the invention of plastic was a primary innovation, while its use in cars or households led to derivative innovations.

    Primary innovations will eventually ‘trickle down’ to reach applied research, which is responsible for the creation of derivative innovations – everything from plastic mop handles to microcomputers and the iPhone cover options.

    I tried to visualize this with the above cube published in 1990. But I am still struggling with the concept myself.

    Oscar, I agree with you that the marketplace is the most important factor in determining the kind of derivative innovations that applied research will create.

    In fact, if there is market for certain types of plastic such as freezer or garbage bags, applied research would never have made them as common as they now are.

    But all this suggests that crowds are rarely if ever the root of such smart and innovative uses of derivative technology such as Twitter or SMS messaging to mention two examples. But for fine-tuning as part of a focus group crowds can be invaluable.

    Thanks again for posing these intriguing questions and challenges. I look forward to your next comment.

  • http://My.ComMetrics.com Urs E. Gattiker

    Oscar , I agree, the line b/w primary and derivative innovation is not always clear cut.

    I have tried to address this in the innovation cube shown here: http://cytrap.eu/files/ComMetrics/2009/image/12…

    Primary research is what leads to primary innovation and is based on work evolving out of the testing of theoretical relationships. Hence, the invention of plastic was a primary innovation, while its use in cars or households led to derivative innovations.

    Primary innovations will eventually ‘trickle down’ to reach applied research, which is responsible for the creation of derivative innovations – everything from plastic mop handles to microcomputers and the iPhone cover options.

    I tried to visualize this with the above cube published in 1990. But I am still struggling with the concept myself.

    Oscar, I agree with you that the marketplace is the most important factor in determining the kind of derivative innovations that applied research will create.

    In fact, if there is market for certain types of plastic such as freezer or garbage bags, applied research would never have made them as common as they now are.

    But all this suggests that crowds are rarely if ever the root of such smart and innovative uses of derivative technology such as Twitter or SMS messaging to mention two examples. But for fine-tuning as part of a focus group crowds can be invaluable.

    Thanks again for posing these intriguing questions and challenges. I look forward to your next comment.

  • http://My.ComMetrics.com Urs E. Gattiker

    Oscar , I agree, the line b/w primary and derivative innovation is not always clear cut.

    I have tried to address this in the innovation cube shown here: http://cytrap.eu/files/ComMetrics/2009/image/12…

    Primary research is what leads to primary innovation and is based on work evolving out of the testing of theoretical relationships. Hence, the invention of plastic was a primary innovation, while its use in cars or households led to derivative innovations.

    Primary innovations will eventually ‘trickle down’ to reach applied research, which is responsible for the creation of derivative innovations – everything from plastic mop handles to microcomputers and the iPhone cover options.

    I tried to visualize this with the above cube published in 1990. But I am still struggling with the concept myself.

    Oscar, I agree with you that the marketplace is the most important factor in determining the kind of derivative innovations that applied research will create.

    In fact, if there is market for certain types of plastic such as freezer or garbage bags, applied research would never have made them as common as they now are.

    But all this suggests that crowds are rarely if ever the root of such smart and innovative uses of derivative technology such as Twitter or SMS messaging to mention two examples. But for fine-tuning as part of a focus group crowds can be invaluable.

    Thanks again for posing these intriguing questions and challenges. I look forward to your next comment.

  • http://My.ComMetrics.com Urs E. Gattiker

    <img style=”font-size: 16px;vertical-align: baseline;background-color: transparent;float: right;padding: 0px;margin: 10px;border: 0px initial initial” src=”http://cytrap.eu/files/ComMetrics/2009/image/12/2009-12-19-Nature-of-Innovation-Cube-by-Gattiker-1990-p-21.gif” border=”1″ alt=”Image – Innovation Cube by ComMetrics – invention of plastic was a rimary innovation while its use led to derivative innocations” width=”275″ height=”250″ />Oscar , I agree, the line b/w primary and derivative innovation is not always clear cut.

    Primary research is what leads to primary innovation and is based on work evolving out of the testing of theoretical relationships. Hence, the invention of plastic was a primary innovation, while its use in cars or households led to derivative innovations.

    Primary innovations will eventually 'trickle down' to reach applied research, which is responsible for the creation of derivative innovations – everything from plastic mop handles to microcomputers and the iPhone cover options.

    I tried to visualize this with the above cube published in 1990. But I am still struggling with the concept myself.

    Oscar, I agree with you that the marketplace is the most important factor in determining the kind of derivative innovations that applied research will create.

    In fact, if there is market for certain types of plastic such as freezer or garbage bags, applied research would never have made them as common as they now are.

    But all this suggests that crowds are rarely if ever the root of such smart and innovative uses of derivative technology such as Twitter or SMS messaging to mention two examples. But for fine-tuning as part of a focus group crowds can be invaluable.

    Thanks again for posing this intriguing questions and challenges. I look forward to your next comment.

  • Pingback: Ralph Kerle

  • Pingback: Urs E. Gattiker

  • Pingback: tim gier

  • Pingback: Johnson Lab

  • Pingback: TBO interactive

  • Pingback: Onno Bruins

  • Pingback: André Engelhorn

  • Pingback: Innovation Sociale

  • Pingback: ComMetrics weekly review: Do mind the GAP » social media marketing, social media monitoring, Twitter monitoring, customer relationship management, usability test, best practice, checklist » Helping you benchmark smarter to increase ROI with: Social medi

  • Pingback: 4 strategies to leverage usability tests » best practice, checklist, social media monitoring, social media marketing, benchmark testing, Twitter monitoring, Facebook strategy, customer engagement » ComMetrics University social media seminar (Weiterbildu

Previous post:

Next post: