Google’s Website Optimizer: Optimization for the Masses

I’ve been receiving lots of calls the past 24 hours about Google’s “release” of their multivariate testing tool called Google Website Optimizer. Here are my thoughts:

• Since you need to have Google Analytics to use the tool this immediately places the tool as a small to mid-market product. I’ve spoken about analytic platforms with hundreds of leading businesses and not a single one of them has Google Analytics or would ever consider using it.

• The tool only does full factorial testing. Full factorial tests require LOTS of data to achieve confidence. This means lots of clicks and spending lots of money on ads to get the conversions needed to reduce margin of error with the results. This is great for Google but not so great for its users (see above) who likely have limited ad budgets. It could take some customers months to see results from a test. In dynamic markets and ones with seasonality this could render some results worthless…after waiting so long to get them. This is the big mistake Google has made with this tool. Had Google used Taguchi methodology and fractional factorial tests along with less data needed for confidence they would have been able to offer the speed and efficiency that I believe comprise the best testing methodologies that are based on sequential improvements via iteration.

• Testing is better than not testing. However effective testing and optimization is a practice. You need to know what to test, where to test, and how to test in order to get the best results. This is partially comprised of a certain design aesthetic around testing and creation of test elements, an understanding of users and their goals and pure marketing acumen. Google says they offer support through their partners but I wonder what type of support. I hope there will be a mass of people that get really interested in testing and optimization. They will only stay interested if they get results.

• The tool only handles site side multivariate testing, not AdGroup title and description testing.

• The tool only handles single page tests. Not really sure why they call it “website optimizer.” Web page optimizer is more like it.

Google has been working on this product for at least a year. I actually heard it was going to be released at SES San Jose back in August. I’m kind of surprised Google didn’t make a bigger hoopla about it and announced it at E-Metrics. It seems to me like the announcement was a knee-jerk reaction to Yahoo’s announcement of Panama and the targeting and quality score features that will help optimize campaigns. I’m also surprised at the relative lack of interest in the announcement in the search world.

Overall, I think Google’s entrance into testing and optimization is analogous to Ford’s introduction of the Model T. Google Web Optimizer is a basic, utilitarian testing tool for the masses. The same way the Model T grew the market for autos, a larger market for testing and optimization helps users, businesses and even us optimization guys. I wish them success because that will mean lots of people are testing and optimizing and the web is becoming a better, more relevant place.