The Must of experimentation as a process

Today continue to learn more about the process of creating a test, with mentor Peep Laja, founder of CXL Institute. Without a doubt, learning about the process in which we must make a correct conversion optimization is one of the key pieces of the degree, as well as when thinking about growth marketing.

Last week we were talking about the different frameworks or step-by-step to make an a/b test, taking into account the priority of the test, the relevance in the business, the difficulty to implement it and in general the time and total impact of it in the achievement of business objectives.

However, this week we go back a few more weeks in the process to understand that a conversion optimization analysis is not part of a series of ideas, which are entered into a “framework” and after obtaining the result of priority is tested. It’s much more complex than that, without a doubt.

Peep makes a very interesting analogy to understand how to approach a conversion optimization process. When a patient goes to the doctor and tells about his problem or pain, he does not receive an answer from the doctor: “we are going to go immediately to a surgery room, where we are going to open you up and examine the origin of the pain you have”. None of this. Doctors, just like marketing experts, must have a process for investigating the pain or the origin of the problem beforehand.

This point of view is undoubtedly very interesting when it comes to optimizing the conversion rate of a page or a product, as well as the scientific approach to the process of improving a specific problem. Without a doubt, before attacking the evident problem, a rigorous investigation must be made in which the origins of this problem are sought to be answered. Afterward, hypotheses that can solve the problem will be sought, a test will be run to evaluate it, the results of the test will be available, and finally, if it is successful or if we learn from it, it will be implemented.

The process is summarized in the following steps: Process experimentation

Conduct Research

2. Build Hypothesis
3. Create Treatment
4. Test treating ideas
5. Analyze results
6. Follow up experiments
Again and Again, it’s the key to the process

Conversion optimization is 80% research, 20% implementation.

The first of the points, “Conduct Research”, has a series of points that must be attacked, so that there is a procedure to follow to evaluate the possible symptoms of the problem. First, we found that an analysis should be done at a technical level which would evaluate vital points such as:

1. Technical analysis (lowest hanging fruit)
a. Test technical functionality to:
i. Browser
ii. Device
iii. Screen size
iv. Page speed.

Many companies do not take the time to evaluate whether their problems are caused by technical problems in their application or on their website. In this sense, many products change over time, adding more value. However, many times the “product managers” do not stop to see the potential errors that your product or your website may have for different devices (Mobile, tablet or desktop), browsers (safari, Chrome etc…), screen sizes from where the digital product is consumed, as well as the speed of the website.

A point I would like to add here is that normally the information is given by tools like Google Analytics, where you can find (if well implemented on the site), all possible information on the website. This, without a doubt, generates a lot of data and in case of not having clarity at the time of searching, it is very likely to get lost in the interpretation of it and not generate any contribution. In this sense, it is necessary for an initial stage, to understand the variables that are more relevant, and as this knowledge is learned more and more, to find relationships between multiple variables.

Finally, it is also important to highlight that it is not always necessary to analyze all the information, of the whole website. This, given that often only 5% or less of the website, is the one that generates more traffic and therefore the highest conversion. Without a doubt, the navigation “path” of the website must also be analyzed like this, to understand if all the elements that are within the page are directing towards the required objective or on the contrary, generate multiple exit points for the site visitor.

After the technical analysis is done, we move on to the heuristic analysis, where we analyze variables such as:

1. Heuristic analysis
a. Friction
b. Distraction
c. Motivation
d. Relevant

The heuristic analysis starts with understanding the experience and behavior of a visitor to our site or product. In this sense, it is possible to make the analysis initially from a very small team, or if you have sufficient resources, make this analysis with the target audience you want to reach.

In line with the heuristic analysis, we evaluate the friction that may exist in the web page, the different elements or points of distraction that each page may have, the motivation that the web page invites, that is to say, the sense of the text which invites to a particular action. And finally, the relevance that all the elements and content of the page have, directing the audience towards a particular action.

Now, once the technical analysis and the qualitative or heuristic analysis of the most relevant pages of the website or the product has been achieved, it is necessary to move on to the analysis of the “digital Analytics” part of the process of defining metrics, which takes into account variables such as

1. Digital Analytics
a. Is everything measured?
b. Is the data accurate?
c. Where are the leaks?
d. Correlations
e. Segmentation

What Digital Analytics does, is to evaluate if all the elements that have relevance in each of the pages are measured or not, is data that can be reliable? That is to say, is the tool we are going to use for analysis well defined and installed?

After that, it is correct to ask ourselves where are the different leakage points of our pages, what are the correlations between the behavior of our visitors and the achievement or not of our main objective, as well as if it is possible to make a granular segmentation to understand what kind of audiences we have in our site and what is their behavior in particular.

So far, many interesting questions have been solved thanks to Peep’s knowledge, who very generously emphasizes that we must have a clear process to evaluate the conversion or optimize it on our website. Undoubtedly, many times marketing people do not take into account these processes that directly affect the conversion of objectives on our website, if not, on the contrary, are given as existing or correct, but when implementing it is seen that this is not right and therefore the company does not reach the results expected.

I will return next week with more knowledge very interesting and I must confess that I must see this module several times, put into practice in the enterprise where I work, and certainly return to learn more.