Measuring the Success of Non-Commerce Websites
I just had some coffee. I feel better.
Alex Langshur (PublicInsite) is here to talk about how to measure success when it’s not about sales and shopping carts. I’ll admit; I’ve been looking forward to this one since I got here. I hope it delivers.
Alex jokes that his session is up against Avinash Kaushik and Jacob Nielsen so he has a bone to pick with Jim Sterne. Heh.
Alex starts off saying that there are plenty of institutions where dollars are not part of the equation. They’re working a lot with Harvard University. Harvard is struggling to find the value of the online channel to their organization and they’re trying to help them with that.
Every time they try and quantify the value of the online channel people are still looking at shopping carts. It’s like trying to shove a square peg into a round hole. What’s interesting is that sometimes you can actually use a square peg to fit the round hole problem. Like with square watermelons!
Engagement: It’s a buzzword but we’re stuck with it. If you go and look at Eric Peterson’s engagement equation, it’s still a number. And numbers don’t necessarily help you manage all that well. It’s one way of approaching it but he’s not going to focus on it.
What should I measure?
When you try and reduce really complicated sites that are an expression of the organization into a simple number, it doesn’t work. You have to find another way. There are two things you need to look at:
Pre-click behaviors: These are leading indicators like visibility and alignment
Post-click behaviors: These are lagging indicators like conversions and intensity.
Pre-click is going to drive post-click behavior. You have to use benchmarks in order to give these things value. If you’re not using benchmarks because there’s “nobody like you”, you’re missing it.
Benchmarks are really important because they give you perspective. It helps you to understand potential audience size and where you sit relevant to others. Excellent free sources include Google Analytics, Compete.com, Quantcast,com, index.fireclicks.com and your own analytics vendor.
Visibility: What affects our online profile? SEO (recommends using Yahoo Site Explorer to check backlinks), traffic acquisition campaigns, buzz monitoring (he recommends Trackur). Who’s talking about you, where, when & what is being said?
Alignment: What is our alignment to the audience and its needs? Look at the intent. What keyword themes bring visitors to the site? Look at your search analytics. Also pay attention to audience segmentation. Are we reaching the right audience? Are we tracking the right audience? Can we segment, and if so, how?
When you’re doing keyword research, there are a million of opportunities for segmentation and alignment in the long tail. The head may be your bread, but the long tail is your issues. They’re the authors on your Web site, the books you’re publishing etc. You need to look into it.
You do that by reverse engineering the Information Architecture and breaking the long tail terms into categories. You can use text mining tools to help you, but you really have to create the categories and themes by hand.
Another part of alignment means asking yourself if you’re getting the right people. Do these 1500 visits you’re getting from Europe influence the key site metrics? If they’re not aligned to you, those visitors degrade your stats and skew your metrics in a negative way. You must find a way to address that. It may mean playing with your geo-targeting.
Intensity: What is the change in key performance indicators for visits? There are a wide range of KPIs to choose from. Things like page views/visits, average visitor duration, unique visitors, user generated content volume, etc. Don’t track them but all figure out which ones matter to you. The Web Analytics Association has a big book of KPIs. You can reference that. If you go beyond 15 KPIs, you have too many. He prefers to go down to 10. When you’re dealing with non-commerce based organizations you have to crystallize the mission or you risk losing the discussion when people become too tied up with the numbers. Don’t design the indicators by committee.
Conversions: How are value events performing? Things like email sign ups, RSS feeds and clickthroughs, downloads of key content, AJAX/Flash events, Send to a friend/print pages, social tags and bookmarking, and satisfaction survey results. In a Web 2.0 world you can use tools like Feedburner to track value events.
How should I report?
- Don’t provide access to raw data, reports or the tool
- Find the hook that can start a conversation, then build the story
- Take the time to report less and inform more
- Contextualize data in terms that matter to the recipient
- Look for champions in the organization
- Nothing succeeds like success
Case Study: Public Sector, health promotion
Goals: Encourage visitors to inquire about healthy eating and inform them of the benefits of healthy eating
Desired Outcome: Encourage them to take action that reflects a shift in their thinking about that subject.
Visibility: Find actions that would raise awareness – SEO
Alignment: What is KW analysis saying about intent?
Intensity: How are they using the site?
Conversions: What are the outcomes?
Visibility: If they hadn’t raised visibility, they would have had a very low amount of traffic. It was one of those things where you can do great stuff, but if you can’t get people to come to the site, no one will know about it. And if that happens, you won’t get the money to do it again in the future.
Alignment: They were looking for people who were at home and people in the K-12 segment. They had 20 percent of their audience from those segments. It was a lot higher than previously.
Intensity: People were coming but they weren’t coming back. Wasn’t a lot of intensity of use. That was a problem going forward.
Conversions: Terrible conversion ratio of .26…but it was double last years. They had a fairly respectable sign up rate. How do you report that?
Created a color-coded scorecard that explained the define objective, the result and a color-code that said if the objective was met. The purpose was to start the conversation. They knew if they gave them a report no one would read it. They didn’t send it by email. They printed it and distributed it by hand.
Question & Answer
Do you have any preferred methods for doing long tail keyword analysis and putting things into buckets?
The short answer is no. They’ve run studies looking at the top 100, 200, 300, 400, etc keywords. The curve is like a normal distribution where once you go beyond 600 keywords, the deltas are under a percent. If you do the Top 500 you’re going to be capturing the major portion of what the intent is. If you still to that, you’ll have a good understanding as to what’s going on.
You talk about profiling segments. It’s hard to do in practice. How have you gone about profiling segments?
- Geographically profiling: You HAVE to be doing this.
- Profile-based navigation
- Qualitative analysis
How do you establish ROI?
In a public sector org, ROI is meaningless because you can’t bring it back to profit and loss centers. In a nonprofit org, it might be a little bit easier but it’s tough to do. He believes in establishing ROI based on programs and initiations. ROI is “are you reaching the right target audience?” He goes back to those kinds of value adds.