Web Analytics Roundtable

Hi all. I has a question: Why do conference organizers always schedule Web Analytics sessions first thing in the morning on the last day of the conference? Don’t they realize how tired and nonsensical I am at this point? Seriously. Think of the bloggers! Or maybe just think of me.

Jim Sterne (Web Analytics Association) is moderating this one where Gary Angel (Semphonic) will be the sole person presenting. Wes Funk (Omniture), Brett Cosby (Google Analytics) and Richard Zwicky (Enquisite) will be on hand for the Q&A.

Now for Gary Angel.

[I wish I had my glasses on. It’s way dark in here. Also, it smells like hash browns. Mmm, grease.]

Gary starts off saying that there are two main "flavors" of Web analytics: Software-as-a-Service (SaaS) systems implemented using tags and software you can run in-house that processes files called Web logs (and sometimes tags). Each of these can provide similar data and capabilities.

Tag-based SaaS systems tend to be easier to implement and have better data-quality out of the box. If you have a lot of internal data about visitors (from a gated site, for example) it can be easier to integrate with software you run in-house. The most popular systems on the market are SaaS-based.

Gary notes that Web analytics data is notoriously unreliable and that even at its best is never a "system of record". The idea that "trending" data protects you from data quality problems is only a half-truth. He thinks all data is extraordinarily messy, though, it’s not just Web Analytics.

Key Data Quality issues include:

• Tracking visitors over time
• Knowing how long visitors spend on each page
• Knowing where visitors go when they leave
• Counting robots – automated agents as real visitors

Web analytics systems will never reconcile.

What Really Matters With Web Analytics Tools?

There’s no one tool that you should definitely use, it will depend on your personal needs. It will be your measurement needs that drive real world requirements. Some of the key factors that really matter are:

Visitor Segmentation: Things to consider when selecting your tools:

• Can Segments be created without tags?
• Can full logical operators be used to define segments?
• What data can be used to define segments?
• Can external data be used natively and combined with Web data in segment creation?
• Can Segments be created via data-drive techniques like neural networks?
• Can Segments focus on visit or visitor behavior?
• Can Segments be defined based on time and event sequences?
• Can distributions be produced on key behaviors to assist in segment creation? (How many came to your site once, twice, three times, etc…)

Segment Methodology

• Are Segment samples or against all data?
• Are segments created in real-time or delayed?

Dimensional Reporting: The ability to take one variable and look at it against another variable. N-Way Cross-tabulation (Viewing the counts of variable by one or more other variables) is an ESSENTIAL part of analysis.

Key Dimensional Reporting Capabilities:

• Visitor and Visit Distributions on variables
• Cross-Tabulation of all variables in Reporting
• Multi-Dimensional Tabulation of variables for Analysts
• Ability to export N-Dimensional Tables to Excel
• Ability to apply visitor segments to N-Dimensional Views
• Ability to distribute N-Dimensional Views once created
• NO DATA CROPPING on high-cardinality variables like PATH and SEARCH KEYWORDS

Management Reporting: Every online business spends time on management reporting. It is an essential element of telling the business story to key decision-makers. Ask yourself if you can get the information out of this tool easily and in a way that you can distribute it to your organization. Some tools make it easy for you to do this; others make it a lot harder. Find out what your tool can do for you before you buy it.

Key Management Reporting Capabilities:

• Ability to combine and tailor views of the data
• Ability to export data to Excel flexibly
• Near real-time segmentation that integrates with Excel Automation
• API to the reporting data

Setup: Building a tag is not rocket science. Many Web analytics packages do require a fair amount of work on the tag if you want to take full advantage of their system. The more things you have to put in the tag, the less flexibility you have and the more tedious it is to fix when you make a mistake.

Key Setup Capabilities:

• Ability to create the most complete analysis (segmentation, campaigns, funnels, hierarchies) without tag changes.
• Light-weight tag
• Ability to capture data that is available only in real-time in custom variables
• Ability to capture customer identification and use it for data integration

Data Integration (online AND customer): Most businesses will ultimately need to combine online and offline data AND Web site and other online data. Think about what kind of data you’re going to need. Not everyone does email or banner ads. Think about your own specific data integration needs and press your vendor about what they offer.

Key Integration Capabilities:

• Ability to integrate with key online systems include email, paid search, competitive analysis, and banner systems.
• Ability to provide a data feed back to the client for customer integration.
• Open architecture, Web service, API, or other seamless data access.

SEM Capabilities: Search engine marketing has its own unique demands and requirements. Here are some of the key points if your primary Web analytics interest is in search.

Key SEM Measurement Capabilities:

• Track results by both actual search term used and search term purchased
• Track content match scores
• Day parting and time parting in the web analytics reporting
• Flexible attribution models
• Cross-attribution reports (how much of Campaign X overlaps with Campaign Y)
• Ability to collapse search terms and analyze them as a unit (important for analyzing the tail)
• Side-by-side performance of SEO and PPC
• Cross-Tabulation of geography by keyword

Additional SEM Measurement Capabilities:

• Tracking creative in the web analytics tool
• Ability to show common “combined” search terms (Term X then Term Y entry)
• Ability to Path over time at the Event Level
  • X visitors entered on PPC Search – Y re-entered on visitor search – produced a success
• Over-time report (visitors who entered on PPC during July did what in August, September, etc.)
• Page performance by entry type report (SEO, PPC, other campaign, direct, previous page)

Key Concept: Most tool evaluations focus on things that turn out not to matter at all when you actually have (and use) the tool. Pay attention to what you really need, and evaluate in-depth.

Summary: Thinking About Fit

How important is Web analytics to you? It’s a function of how important the Web site is. You can’t have a great Web site without Web analytics, but you can have a satisfactory one. Some tools demand that you invest more (in time and money). You need to decide how likely that investment is to pay off for you.

How to get started: Think about your organization, culture, and knowledge. Choose a tool/resource direction that is realistic. Take the time to build a roadmap of what you want to accomplish

What you should worry about:

• Getting people is hard
• A good implementation is harder than people say
• Web analytics won’t happen without both tool and resources
• The market is immature and there are no safe approaches
• What are you getting back — if you don’t demand interesting analysis, you won’t get any.

Question & Answer

Why should I spend money on a Web analytics tool when I can get one that’s free?

Wes: Sometimes the data you’re collecting is fairly deep and it needs to go and connect different data systems. In these cases, not only is it important to easily see the user flow from an email, to a search term, and to your site (and how it might interact with a CMS), but you also want to get the next generation of that interaction, beyond those corollary relationships. You want to be able to act. When someone comes in via an email campaign, you want to recognize that in your CMS. You want to understand that this person answered a survey that came from another partner. You get stronger data connections so you can action them and treat users in a more holistic manner.

[So what we’re saying is that Omniture is more sophisticated and Google has trouble with relationships? – Lisa]

When do I need to move to a more sophisticated tool?

Richard: Seventy five percent of our users also use GA and another 20 percent will use Omniture. He says that the features in Gary’s analytics wish list are actually offered by his company.

Gary: Search marketers are spending a lot of money. If you can get a relatively small advantage from a software solution, it’s worth it. When you’re spending money it’s important to think about how you can leverage these products more effectively. He thinks there are a fair number of companies where Google is more appropriate and there are companies where Omniture is more appropriate, regardless of the cost. Anyone who uses a Web analytics tool well can improve their search engine marketing program. Economic figures into every decision, but you also have to look at the benefits side. Money should not be the deciding factor in which tool you use. You have to use the right tool for you.

Do you have recommendations for how to communicate to Analysts who still talk in terms of hits?

Jim: Acronym for hits = how idiots track success.

Wes: Ask them the questions that they should be asking and help them understand it’s not just about hits. Ask the questions that are going to drive success.

Richard: It’s an educational process. Give them a number and then explain the context for them.

Brett: If you can show someone a very simple equation and how it makes them more money, that’s the way to do it.

Gary: He believes search marketing people will struggle with this less than other people. He says that senior execs know what matter to the business, it’s the analysts that don’t. Search engine marketing has helped people sharpen their focus to Web analytics because it is about marketing and the dollar. You have to talk dollars and what matters to the business.

What are your thoughts on Nielsen’s move towards time on site? Will engagement ever be anything more than a vague idea? Will it be quantified and what will that look like?

Brett: That’s the exact opposite of what Google is trying to do. We want to get people in and out as quickly as possible (Apparently no one introduce Brett to Universal Search…). As far as the term "engagement", it’s such a vague term. Nail down what that means. What is the goal of your Web site? You want to define that goal very clearly and get people to do that.

Wes: They’re about helping people to ask the right questions and finding the segment that is the most interesting to them. That’s what they want to action. Is it time on site? How fast users get off the site? What part of the funnel are you looking at and what are you trying to optimize?

Gary: Not every company has to care about branding, but it does matter and it’s important to find a way to measure what matters. You can’t just rely on traffic. That’s not a good measure of a program’s effectiveness. If you’re trying to build branding, you have to focus on engagement. It’s not about traffic.

Could each of you give one example of where a customer used their Web analytics data to make $50,000.

Richard: They have a customer who sells tours through Africa and they used the geographic segmentation feature to find out where users were coming from. They found out a lot of users were coming from the UK and then made changes to their site that incorporated British English to target them.

Brett: He has a company in the surf apparel industry. He was looking at this geographic map in Google Analytics and found that sales were dropping. His analytics showed him that everyone from the middle of the country to East had suddenly dropped off! It showed him that they had misconfigured the server to block all IP addresses east of the center of the country. Heh. Another way is that you have clients who look at bounce rates. With just a bit of tweaking you can really increase the number of conversions.

Wes: When you notice that the search term came from Google Spain, you may not be able to put your whole site in Spanish, but you can add a banner attracting and engaging those users.

Gary: Natural search people generally don’t convert nearly as often as people who come directly into the site. We see a lot of people go through fairly expensive natural search programs and then not get the returns they expect. We did an analysis of people coming in from SEO terms and looked at what they did next. They realized that they were coming into different places in the site than other users. They were going directly to article pages and were missing out on all the engagement factors. What they did was add some of the engaging stuff into the article templates so that natural search people would see it. Natural search is a huge driver of traffic, with just a few tweaks you can take advantage of that and get some of that traffic to stick.

Gary also had a client who had 600 Web sites to manage. They wanted to be able to identify which sites to get rid of. They focused on the true value of the Web sites and found which sites were worth keeping and which they could drop. Saved them lots and lots of money.

Wes, in your role as Online Marketing Director, you’re actually doing this stuff. How do you value your search traffic? What are you looking at?

Wes: How many people do we creatively force? How many download a white paper or attended a webinar? One of the important things we’re doing is looking at our keywords and understanding what groups of keywords do what things. That’s really constricting. We’ve done to SEM what Gmail did to Email. Instead of filing things, they’re tagging terms with different attributes (ie. this is a branding keywords and it’s for Product X). If you have all of those keywords tagged with one of those attributes, we can then go in and see how much you spend on your brand keywords. How much money are we spending on a certain product?

How can we best integrate Web analytics information with offline marketing dashboards to show how our offline activities correlated with online activities?

Brett: The easiest way to do this is use a unique landing page for offline stuff. Allows you tell which ads are having an effect. A lot of times you’ll see an increase in traffic anyway since not everyone will go to that specific URL. If you’re running a newspaper ad or a TV ad in a specific location, you can look at your data to see if visits increased in that location. This is a good way to test creatives.

With offline, one of the interesting things is that you can’t be telling people to come down to our dealership this weekend. You have to tell them to go to your Web site to perform an action so you can measure.

Lisa Barone is a writer, content marketer & VP of strategy at Overit Media. She's also a very active Twitterer, much to the dismay of the rest of the world.

See Lisa's author page for links to connect on social media.