Customer Research + Analytics
This week I continued with my studies of Growth Marketing for Conversion XL. I am currently learning a lot from Mercer, a specialist in the analytics module. However, I decided not to advance in the line of analytics until I complete the previous certifications that are pending.
So far, I have User-Centric Marketing, Conversion Research, and A/B Testing certifications pending. Of these, I have the most knowledge and skills in Conversion Research, so I decided to start studying this chapter again, to take the exam and pass it.
Heuristic Analysis in which you want to start with a heuristic approach to get familiarized with your website set several challenges and later validate or refuse with data.
We have to be aware of some biases:
● Bias blind spot: a tendency to be over-sure about your point of view.
● Confirmation bias: favor information to confirm your beliefs.
“If you can’t explain what you are doing as a process, you don’t know what you are doing”.
Steps for heuristic analysis
Here are the steps that I use for performing a heuristic analysis of a given website.
I start by conducting thorough walkthroughs of the site with all the top browsers and each device category (desktop, tablet, mobile). I pay attention to the site structure, go through the checkout/form filling process. The goal here is to familiarize myself with the site and its structure and to identify any cross-browser and cross-device issues. Read the chapter on walkthroughs.
When evaluating a site, I will:
● Assess each page for clarity — is it clear and understandable what’s being offered and how it works? This is not just about the value proposition — it applies to all pages (pricing, featured, product pages, etc).
● Understand the context and evaluate page relevancy for visitors: does the web page relate to what the visitor thought they were going to see? Do pre-click and post-click messages and visuals align?
● Assess incentives to take action: Is it clear what people are getting for their money? Is there some sort of believable urgency? What kind of motivators are used? Is there enough product information? Is the sales copy persuasive?
● Evaluate all the sources of friction on the key pages. This includes difficult and long processes, insufficient information, poor readability and UX, bad error validation, fears about privacy & security, any uncertainties and doubts, unanswered questions.
● Pay attention to distracting elements on every high priority pages. Are there any blinking banners or automatic sliders stealing attention? Too much information unrelated to the main call to action? Any elements that are not directly contributing to visitors taking the desired action?
● Understand buying phases and see if visitors are rushed into too big of a commitment too soon. Are there paths in place for visitors in different stages (research, evaluation, etc)?
Usability Evaluation
Jakob Nielsen defines usability as:
Learnability: How easy is it for users to accomplish basic tasks the first time they encounter the design?
Efficiency: Once users have learned the design, how quickly can they perform tasks?
Memorability: When users return to the design after a period of not using it, how easily can they reestablish proficiency?
Errors: How many errors do users make, how severe are these errors, and how easily can they recover from the errors?
Satisfaction: How pleasant is it to use the design?
Use Checklist
Task: Open your website in one window and the other opens the usability checklist. Write in a spreadsheet every issue that you found. Implement every issue-solution according to the prioritization defined.
Survey Design Theory
Qualitative survey approach: Must be done by a zero-sum analysis. Get all the qualitative answers and make clusters according to keywords or insights found. Then do a Quantitative analysis according to the qualitative data.
Bouncing betas: When research is made to a very small audience, not all the customers are going to fit in the research, so the answers would be 0.
Error in surveys: Mixing behavior questions with attitude questions. Others are questions that don’t communicate. Surveys too long (5 to 10 minutes long max.) An error of central tendency, fatigue increase, and people answer with neither agree or disagree, but you can’t go further in the analysis.
Selective Perception > Something customers agree with you, tent to automatic agree
Survey Customers via Email
Important to send out a purchase as soon as possible, from the point that your customers purchase your service/product.
From 8 to 10 questions max, avoiding the fatigue of customers.
Usability Testing Vs A/B Testing: The difference between both options is that usability testing shows what issues are causing the user problem or friction to accomplish a goal, against a/b testing that shows the probability that A option is better than B, with a statistical significance. In the usability test, you only need some users. In an A/B test, according to the number of web visitors, you need a specific amount of them to validate a hypothesis.
The way to create a test on our websites is to make a Usability test to find the problems on the website. Later, create a hypothesis, make an A/B test and get a result in your hypothesis.
Mouse tracking
Is useful to identify:
● Where people click and where they don’t
● How far they scroll on any given page
For a Head Map analysis, it’s possible to use tools like “algorithm tools”, but it's necessary to take into consideration that these instant head maps are created by machine predicting algorithms, so there’s no attribution to real end-users.
Google Analytics Health Check
If I have a service fee for a google analytics health diagnosis, It should create trust with customers and also, people want fast deliveries on their works. So It could work great.
The first thing is to check for the needs of the company, tracked in GA. Ask several questions that are important:
● “Does it collect what we need?”
● “Can we trust this data?”
● “Where are the holes?”
● “Is there anything that can be fixed?”
● “Is anything broken?”
● “What reports should be avoided?”
Its well used and recommended to add a goal when there is any error pop on a form or a checkout page. This way, with “revers goal path”, you can check how many goals are completed with this mistakes, so you need yo be able to fix them and see a decrease in the total mistakes.
A/B Testing Mastery Course
Speaker: Pepp Lajaj
Type of experiments:
● Lift elements: Just delete some elements on the page that doesnt have value to your users and are negatively impacting on your website.
● Optimization: Lean deployment, is the best way to A/B Testing some elements.
The ROAR Model
- Risk
- Optimization
- Automatization
- Re-Think
If you doesn't have at least 1,000 goals per month, you cant create A/B tests oriented to goal conversion optimization.
Which KPI to Pick
From a mature perspective, you might select a KPI in importance from top to bottom if you are a mature company.
● Potential Lifetime Value
● Revenue per user
● Transactions (at least this, if you want to focus in a more business approach)
● Behavior
● Clicks
What can be optimized?
● Customer behavior study: Start looking at what your customers want, frictions, etc..
○ Get the most important insights into your customer journey
Track your website changes with several tools. Also, we can track the changes of any competitor page to see if there are major changes in the site, so we can test also. If the population is shared with them.
Behavioral metrics for website
- % Light interactions in a website
- % High interactions in a website
- % Low intention to purchase
- % High intention to purchase
What to report when we have these numbers?
- Amount of users in every cluster
- Time for users to move from a cluster to another.
Also, it is important to talk with customers' service or hear a call in order to understand what customers want, and need about our product.
Create modules asking for feedback online. Use at much as possible your current users. It could be the ones who interact with your service or product already.
What type of test can we do to evaluate our assumptions
- Five seconds test (measure users first impression)
- Question test (get users feedback)
- Click test (visualize where users click)
- Preference test (find out what users prefer)
- Navigation test (find how your users navigate in your site)
Google Optimize
It’s important to run an A/B test in Google Optimize as similar as possible to every possibility. So, it’s recommended to create a set of pages: “Original”, “Default” and “Variant”. The original one is going to receive 0% of the total traffic, and the default and variant 50% each. So, in this way, we make sure that the original and test version is as similar as possible, so results are more accurate.
How to calculate A/B test length?
● Why do we have to take a complete week for a test?
Weekdays' behavior affects results compared to weekends. Also, evening effects compare to business hours.
● Why 1, 2, 3 to 4 weeks?
Sample dilution or not
Pace/velocity versus business cycles.
You have to take into account the amount of time that a visitor converts on your website, so you can recognize the complete effect of the experiment on a business cycle of a customer.
Difference between an SRM-Sample Ratio Mismatch, when we design a test with the same amount of visitors (50% 50% split test), and you have a 50.2% for any variation instead of 50%, there is a bug in the test. So there is a formula that enables you to find the mismatch.
Statistics Fundamentals of testing
Statistics is the way marketers can tell if an A/B test is true or false, according to data and more importantly, validates statistically any hypothesis.
Population: all potential users from a group that we want to measure.
Parameters: variable of interest that can be measured.
Sample parameter: it’s a sample of a representative group.
Until next week we will be seeing more of the content of Conversion XL.
