What is Conversion Rate Optimization Research?
As with Landing page Conversion Rate Optimization, general conversion rate optimization, like any other serious process that sets out to measure and improve goals, starts with research. This blog post will talk about the basics of research and some important things to set up your conversion optimization process.
Where does this information about CRO research come from?
Before I continue, I want to let you know most of this information is from Peep Laja, CEO of CXL.com, and, from my point of view, a great mentor and rising startup rockstar. To this effect, I recommend the CXL.com course, and also want to let you know this content is part of my mini-degree studies in Conversion Rate Optimization. A great tool to professionalize and optimize your ROI on any UX-based investment, specifically on web assets like e-commerce, lead generation, and growing your userbase.
So, how does research fit in the process of optimization?
It’s the beginning, the research process is the first part of any optimization cycle, CRO is a specialized-for-web-conversions application of an evolution of the general optimization cycle created by Edward Deming`s “Shewhart Cycle,” which had evolved into Plan-Do-check-Act (PDCA). This process is present in Agile, Scrum, Lean and other methodologies and academic fields, like psychology and product design. All of these follow 4 stages:
The research process in CRO is the beginning of these 4 stages, and it would fit as a subsection of the “Plan” part. ( Just to understand its academic validity). Having said this, it’s also part of its own body of knowledge and has its own very different applications, methods, and effects in the world of e-business. The end of an optimization process would be reaching the “local maximum” which means that no matter what you change in the elements you have control over, the general setup will not improve the results. When this happens your only next step is to create a completely new UX.
1. Do you really need optimization or do you need exploration? What’s the difference?
Optimization helps when you already have customers, exploration is what you need when you are looking for customers.
Conversion rate optimization is specifically focused on optimizing a working business model. Most e-businesses though, start as either an intrapreneurship effort or an entrepreneurship effort, in other words, you are trying to get your first online customers, be it as the first e-commerce intent of an established business (intrapreneurship), or trying to find customers for your “minimum viable product” of a new business model experiment that is yet to be validated (entrepreneurship). If it’s the latter, you are still trying to figure out who your customers, what is your product, and how to sell it. And for that, I would refer you to my favorite book on the matter (Probably the only one I have read fully) The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses
If you do have customers and you want to get more of them or get more money from them (or users that are reaching other objectives and you want to increase these utilities, either by increasing the user acquisition or the utility per user), then the process begins by looking at the working model and finding opportunities to improve. Note here that we are overviewing a specific CRO research process called the “ResearchXL methodology” because, for now, it’s my “alma mater” :). This methodology offers an order or process for the research based on Peep´s experience and his own research.
CRO Research steps of the ResearchXL methodology
Heuristic Analysis
Heuristics are rules of thumb that a person with a lot of experience in building UX and evaluating results might already know. Also, known as best practices. Why, because it’s often a great place to find easy to implement, high impact changes to your UX. It’s an experience-based assessment of your website. When you check out a website looking for these opportunities, Peep Laja recommends looking for the following CRO items and inventory the ones that you believe have a lot of room for improvement:
Clarity – Remember what a user sees is all a user will understand. There are no spaces for intuition or for assumptions, there should be 0 room for misinterpretation or doubt. Or confusion, in this case, less is more. Don´t expect the user to understand what it is you are selling, how much it costs, what you are getting exactly, etc).
Friction: Too many steps, too many requirements, and unnecessary account creation page. Etc. What can you take out of the UX that won´t affect the customer?
Anxiety: Are there things that create negative emotions, like asking for too much information or putting images or wordings that trigger stress.
Distractions: Unnecessary information that will detour your visitor from the conversion funnel.
I – Technical Analysis
Technical Analysis: These are basically things that aren’t working as expected and are impeding your visitor from getting the UX you designed.
- Browser incompatibility
- Device incompatibility
- Broken code
- Broken pages
- Lows-peed page loads ( slow websites suck!).
II – Digital Analytics Analysis:
Digital analytics focuses on using statistics and numeric data to find opportunities that are not obvious using human heuristics. Data-driven companies and data-based decision making is the way of the future and is probably being used by most successful companies in today´s ever-changing, confusing and innovation disrupted environment. Using data, you will try to find gaps in numbers that follow the general statistical quality control guidelines for manufacturing processes (Aso part of Edward Deming´s great contributions). We are looking for a specific data subsection of your process that has strange KPIs; numbers that are not ” statistically normal” (lower or higher than the acceptable statistical variance) of your other purchase levels. These will shed light on possible:
- Technical problems.
- Funnel structure problems.
- Friction points
- Demographic and UX mismatches and higher than average performance.
Make sure your Google Analytics is configuration is working properly. Usually, new G.A. users aren´t aware that a lot of their “page view” is either tracking spambots or yourself and this is making noise. In this respect, I recommend you read “Don´t use G.A. like Mr. Bean, an intro to G.A. strategy“
III – Qualitative Research
Information that paints a picture that makes sense, a story or fable that is backed up by numeric data, but is more of a hypothesis of what is broken and what is working. There are two tools in this process: Surveys and Polls that will give you insights and pieces of the puzzle for you to come up with optimization opportunities. These might be either to get more accuracy from insights you saw in your heuristic or data analysis, or exploratory, where you are still open to the idea that other users might give you insights you don´t see yourself.
Surveys & Pools
We get information from people who are on our website, we put a poll or survey on key pages. We want to understand why users aren´t moving down the funnel, so polls should capture information for every part of the funnel. Peep recommends two types of questions:
Are you ready to make a purchase?
And for people that answer no:
Why are you not ready to make a purchase?
Peep Laja
Post-purchase Survey: This is to understand what works: Why did you buy this, how did you feel, would you recommend this product, what problem are you looking to solve.
User Testing
Ideally, you get people that represent your target audience to perform specific critical conversion tasks and then observe both their behavior and their thoughts (By letting them think out loud and critique the process as they go through it). So you can understand the UX experience, and you can see what is happening. Laja says, and I mostly agree, that the most important thing to pay attention to is the actions, rather than the critique. I personally say you should pay attention to the critique and the emotional tone of the critique. (Where is the “·$!·”$!” subscription button to this amazing optimization blog?).
Mouse Tracking Analysis
It’s possible to create a correlation between people´s mouse movements and their UX. But mouse clicks have the most obvious and transparent insights. There are a few tools that offer free and paid “mouse movement recordings”. I especially like “Hot Jar”. Look for insights in:
- Where do people click
- Heat maps
- Scroll maps
- Session Replays
*Ask me about these if you are not sure how to proceed.
IV – Post Analysis
After you have done this research, you now have a list of problems that you have identified in a spreadsheet and evaluated. Now your job is to create a systemic analysis of your issues to see where to invest for the best results. To do this you can create a spreadsheet where you add in descriptions for each issue found and some kind of priority-coefficient that usually elucidates the cost-benefit of pursuing ( working on, fixing, and validating the statistically valid up-lifts in conversion rates).
Issues (columns) for each issue would be:
- Instrumentation issues: Is it a data/instrumentation issue: Data is wrong or you found a real insight. Yes/no.
- “Test Solution” Issue Or “Unknown Problem” Issue: An obvious problem with an obvious solution, we just need to test OR is it a known problem with no obvious solution, so we need a creative approach, where you invite product owners, designers, and developers to move forward.
- No brainer issues: Is it a “Just do it” problem or is it a “Don´t start without a thorough testing plan” issue?
- Is it feasible to fix: Do we have the resources and time to do this and is it worth it?. Anseres: Worth the investment or Not worth the investment.
*Alternatively we could add only numeric evaluations of these and formulate a self-sorting system, in the same way, JIRA does for agile/scrum teams to polish UX and general software development.
An optimization program´s efficiency
The entire research and test process or optimization program has 3 KPIs you want to always consciously look to improve:
- Testing velocity
- Percentage of tests that provide a win
- Impact per successful experiment
So, ta-ta for now folks! Let me know if I can help you with any related questions or if you are in need of some consulting work.
ABOUT THE FEATURED IMAGE: This image was originally posted to Flickr by dynamosquito at https://flickr.com/photos/25182210@N07/4265771518. It was reviewed on 16 December 2015 by FlickreviewR and was confirmed to be licensed under the terms of the cc-by-sa-2.0.
