The problem in General Analytics
Google Analytics, Mixpanel, and Optimizely are highly flexible, all-round solutions. They are generic solutions, not specifically designed for conversion optimization. Here's what happens.
General analytics is not built for any specific purpose. Mixpanel, for example, is a general-purpose event tracker suitable for many different use-cases.
Use Mixpanel for anything from measuring how your users fair slaying digital monsters, to QAing a back-end fix, or monitoring and tweaking performance metrics.
This might be a good thing, but not necessarily what you are looking for. Most websites are only interested on doing more sales so the only thing that they want is conversion optimization.
The way general analytics helps you with this mission is to provide endless amounts of data, visualized in all the imaginable ways:
The result: too many metrics, graphs, and opinions to make confident decisions. No clear goal to strive for.
Conversion optimization is all about fixing your weak spots — over and over again. However, general data model is not designed for this task.
- No bottlenecks — not knowing the “low-hanging fruits” where you make the biggest wins with the lowest possible effort
- No leading indicators — not measuring the details on how people gradually build interest and desire before taking action.
- No north star metric — not striving for maximum visitor lifetime value.
Measurement is fabulous. Unless you're busy measuring what's easy to measure as opposed to what's important.
Worse yet, some of the the vital stuffis actually broken.
A/B testing has serious issues
Traditional A/B testing is rather trivial: you make a small change to a web page and see how it impacts a single metric on some time range. Then you wait for months to get the results and you are expected to update your site during that time.
Many businesses would be better off if they didn’t run any A/B tests at all.
Author of “Design for Hackers”
Damaged visitor experience
Traditional A/B testing solutions are absurldy large pieces of software. For example, Optimizely is 200 times larger than Volument:
Moreover, these visual A/B testing tools tamper the first impression with a flaky FOOC effect: the original page is briefly displayed before the alternative appears.
These UX problems cause more people to leave your site even before they become aware of the product.