vicodin

Are You Measuring Social Media or Playing Bingo?

by Rachel Happe on April 12, 2011

Last week, Radian6 hosted its first user conference and I was impressed with the content, speakers, and the logistics of the event itself. One of the panels I attended was the ROI panel, in large part to support Ken Burbary and KD Paine but I realized while listening to the confusion and arguments that started that I am a glutton for punishment. I have never really understood why this topic generates so much discussion and confusion. In my mind ROI is quite simple – the investment in time and resources into an activity and the value you get in return. It’s a financial calculation and it is sometimes, but not always, the way to measure the success of a business initiative. Particularly in ambiguous or innovative efforts there are often benefits which justify the expenditure that have nothing to do with financial return. How you justify your business projects depends entirely on your leadership team and what kind of progress and benefits they need to see in order to make the investment. No one can give you that answer unless they are intimately familiar with situation.

So, I dislike this conversation. It seems to go around and around and I am never quite sure why. But I had a bit of an aha moment during the session when an audience member got up, clearly frustrated and looking for specific answers which none of the experts will provide because measurement is so context specific that, in fact, it would be irresponsible to do so.  However, it finally hit me why there is such frustration. People who started down the social media path because their customers were pulling them in that direction are now swimming in information. They need more resources to coordinate and build on their initial successes and to provide that, executives are asking for proof of value. So they go on a hunt to string together the data that will prove that.

The problem? There is a sea of data because online everything can be measured. There is too much data in fact. It blurs and confuses people because it seems like it all should be useful. But it is not. Add to that, there are literally hundreds of benefits that social methodologies can provide an organization but only a handful that will matter to any one executive. You simply cannot start measuring the value of using social tools if you try to build the case from the bottom up. You will be sifting through and rolling up data forever. How frustrating.

The antidote? You have to start with a goal that means something to the person from whom you need support. Once you identify that, the exercise in proving value becomes a bit more obvious. Issues remain in that social methods are typically only one part of what is required to achieve a business goal. Ken Burbary had a great suggestion when that is the case: measure performance of a process before you apply social methods to it and after – and see if it moves the needle. If it doesn’t, that may not be the best use of a social approach.

For those swimming in data and on the hunt for the measurements that matter, the necessarily vague general conversation about measurement doesn’t seem helpful and is not specific or actionable. However, the experts in the space continually reinforce the concept of starting with the goal. If you don’t know what the goal is and are trying to ‘find’ it through the data, you will continue to be frustrated. If you don’t have a clearly articulated goal, a better idea is to start with a hypothesis and see if you can prove it out, if not work on a new hypothesis until you find one that shows a positive return. But please start with a goal to prove not a bunch of data. Data without context makes no sense.

Image by hownowdesign

  • http://rebeccaadenison.com Rebecca Denison

    I love this, Rachel! You said exactly what I was thinking. I heard people grumbling about wanting specific examples, but it just too hard to give any example that could be applicable in more than a few situations.

    Start with a goal and measure against it. Sometimes you have to get pretty creative to measure what you want to do, but there are usually ways to measure what matters to you. If you’re trying to drive engagement, measure it. How? How do you define engagement? There’s no hard and fast rule, which I think can be intimidating, but I think it’s an exciting challenge! I can define engagement or success for myself and for my clients.

    You’re so, so right. Start with the goal, not the data!

    • http://twitter.com/rhappe Rachel Happe

      Hi Rebecca – thanks for stopping by and commenting. It is so different depending on what you are doing. I often wonder what, if anything, we could do to help individuals with more actionable information but it’s one of those topics that requires consultation at some level to get to any practical advice.

  • http://twitter.com/TCoughlin Terence Coughlin

    Always interesting to read the thoughts of a panelist post discussion, Rachel. I attended the ROI breakout and the tweet-stream was as entertaining as the live action. A definite highlight of the conference.

    It must be a bit of a lose-lose situation when volunteering to take on such a topic as social media ROI in front of such an engaged (and naturally curious and opinionated) audience. Some people are clamoring for a nice, neat answer, but if you offer one, others will slam it down if it does not jive with their own personal perspective and context. On the other hand, if you answer with the absolutely valid “It depends”, you risk disappointing those looking for a simple metric.

    You know you are participating in a controversial and messy topic when those in the audience prelude their questions with longish personal ruminations of their own (and if delivered in a British accent, all the better!). We all have something to say on the topic, formed by our own experience, and as such everyone’s personal answer is unique – as it should be.

    • http://twitter.com/rhappe Rachel Happe

      Hi Terence – you are right, everyone has such a unique perspective on the issue and I’m a believer that everyone has a valid perspective (even if they may need more experience with something). I’m beginning to think I should really just stay away from the topic generally since the discussion doesn’t make much sense to me :)

  • Barry Watson

    I like this blog and especially the line – “Ken Burbary had a great suggestion when that is the case: measure performance of a process before you apply social methods to it and after – and see if it moves the needle.”
    So often I see statistics being used to prove a case which can be way out of context. However,I love the simplistic approach of ” We are here today – lets measure where we are tomorrow!”

    • http://twitter.com/rhappe Rachel Happe

      Precisely – I think simplicity wins over complexity in this area most of the time… especially for metrics in areas where people are pretty unfamiliar with the new processes (like social media)

  • http://www.dayngrzone.com/ Dayngr

    Bingo! (Pun intended!) You hit the nail on the head with this one. I believe that the ROI is subjective and specific to each project and not always defined in financial terms – though that is another area that seems to be ripe for debate. You said it simply “In my mind ROI is quite simple – the investment in time and resources into an activity and the value you get in return. It’s a financial calculation and it is sometimes, but not always, the way to measure the success of a business initiative. Particularly in ambiguous or innovative efforts there are often benefits which justify the expenditure that have nothing to do with financial return.”

    Thanks for being with us at #social2011 and thanks so much for all your feedback. It was such a pleasure meeting you.

    All the best,
    Trish (@Dayngr)
    Community Manager | Radian6

    • http://twitter.com/rhappe Rachel Happe

      Thanks Trish – it was great to meet you last week and all of you at Radian6 are doing a great job helping to clarify this issue so thank you!

  • http://profiles.google.com/danieloyston Daniel Oyston

    Nice post. I have recently started in an online community manager role for a new organisation and have arrived to find SM tools/tactics already in place (that’s fair enough). I am now trying to establish meaningful metrics and it isn’t as hard as people think (and I am working for an organisation that doesn’t sell or have financial goals).

    I have simply looked at the Strategic Business Objectives, mapped them to the stated critical success factors, linked them to KPI’s for each CSF and then linked all the tools to the KPI’s (it now looks like an org chart on an A3 sheet).

    Under each SM tool I have listed 3-5 high level metrics. Now people can see, for example, that we tweet, what metrics we track on Twitter, and how it links back up the food chain to help achieve KPI’s, CSF’s and Bus Obj’s.

    I also find that less is more. Don’t drown in the metrics. Be brutal. Just pick 3-5 key one’s. What you will find is that people will either be happy with that or ask for more. If they ask for more, then you have them engaged and interested and can have a discussion with them about metrics, what a bounce rate is etc

    • http://twitter.com/rhappe Rachel Happe

      Perfect implementation and I couldn’t agree more. In this case, KISS (keep it simple stupid) is a pretty good rule of thumb. If the audience for the metrics can’t follow their meaning, they are useless anyway because the audience won’t trust them. And linking them back to the strategic business objectives is how you tell a story through date. Thanks for the great comment!

Previous post:

Next post: