Problems in General Analytics
Google Analytics, Mixpanel, and Optimizely are highly flexible, all-round analytics solutions. However, if you just want to use analytics to improve your webiste then a general solution is not ideal. Here's why:
General analytics is not built for any specific purpose. Mixpanel, for example, is a general-purpose event tracker suitable for many different use-cases.
Use Mixpanel for anything from measuring how your users fair slaying digital monsters, to QAing a back-end fix, or monitoring and tweaking performance metrics.
This might be a good thing, but not necessarily what you are looking for. Instead you are flooded with too many metrics, graphs, and opinions to make confident decisions. No clear goal to strive for.
Conversion optimization is all about fixing your weak spots — over and over again. However, general data model is not designed for this task.
- No leading indicators — not measuring the important details what people do before they convert, essentially making the optimization impossible.
- No bottlenecks — not knowing the “low-hanging fruits” where you make the biggest wins with the lowest possible effort
Measurement is fabulous. Unless you're busy measuring what's easy to measure as opposed to what's important.
A/B testing has serious issues
Traditional A/B testing is rather trivial: you make a small change to a web page and see how it impacts a single metric on some time range. Then you wait for months to get the results and you are expected to update your site during that time.
Many businesses would be better off if they didn’t run any A/B tests at all.
Author of “Design for Hackers”
Damaged visitor experience
Traditional A/B testing solutions are absurldy large pieces of software. For example, Optimizely is 200 times larger than Volument. That's right:
Moreover, these visual A/B testing tools tamper the first impression with a flaky FOUC effect: the original page is briefly displayed before the alternative appears.
These UX problems cause more people to leave your site even before they become aware of the product.