Heap Digital Experience Insights Report User Guide

User Guide for Heap models including: Digital Experience Insights Report, Experience Insights Report, Insights Report

Digital Experience Insights Report: Installment 2: What Funnels Conceal (that you probably weren't aware of) | Heap

File Info : application/pdf, 13 Pages, 399.78KB

heap-digital-experiences-insights-report2

Digital Experience Insights Report
What Funnels Conceal (that you probably weren't aware of)

Key Takeaways

DIGITAL EXPERIENCE INSIGHTS REPORT

Methodology: From August to Oct 2021, Heap's data science experts analyzed thousands of anonymized user funnels and millions of user events to locate key patterns and trends. In addition, we performed qualitative interviews with hundreds of customers to detail their concerns about understanding their customers' journey. Here's what we found.

The Problem With Funnels

84%

of all funnels deliver incorrect or misguided information, leading to misguided goals and incorrect assumptions

Why does this happen?

Unseen Detractors
59%
of funnels miss an interaction that's associated with lower conversion, leading teams to prioritize the wrong fixes

Miscalculation
23%
of funnels miscount converting users as drop-offs, because those users skip steps before converting

Invisible Paths
63%
of funnels contain an alternative path to conversion that isn't tracked, leading teams to misunderstand how users really navigate their site

DIGITAL EXPERIENCE INSIGHTS REPORT
INTRODUCTION
What Funnels Get Wrong
Welcome to the second Digital Experience Insights Report!
Our first report analyzed thousands of funnels and literally millions of user events to figure out what events funnels tend to overlook. For this report, we had some bigger questions:
Can funnels capture user behavior as it actually happens?
How well do funnels capture the twists and turns of real user behavior? The multiple paths to the same goal? The optional activities users take or don't take on the way to conversion?
In Short:
Can funnels tell us how users really behave? If not, how misleading are they?
The answer:
After digging into millions of user events, we found some alarming information: 84% of funnels provide incorrect or misleading information!
© 2022 Heap Inc. | heap.io 3

How is this possible? The answer is that traditional funnels -- which track user behavior step by step, capturing the dropoff rates at each step -- were built for an earlier era, one when websites offered few choices: users had a single path to follow, which they either followed or didn't.
For users, today's digital experience provide more freedom. They offer choices, alternate paths, multiple routes, and enable all sorts of complicated behavior. That's a good thing! The problem is that funnels are an outmoded tool for understanding this kind of behavior.

DIGITAL EXPERIENCE INSIGHTS REPORT

For example: if one is trying to answer any of the following, funnels will always return incorrect or misleading data. (Yes, always! It's how they work.)
· Does performing an optional step (reading product reviews before adding an item to cart, say) increase or decrease the likelihood of conversion?
· How many users perform that optional step and convert? How does that compare to the number of users who convert without performing that step?
· When there are two possible paths to a single goal, which one has a higher conversion rate? How many users take each path?
· In any given user flow, do users skip any steps on the way to conversion? If so, how many users skip steps, and which steps do they skip?
· Do in-app guides help users get through a specific user flow? If so, how many users interact with the guide, how many don't, and what are the relative conversion rates between these groups?
For this Insights Report, we dug through millions of events to see how often funnels mislead teams into thinking they understand user behavior, when in fact they're missing what's really happening on their product or site!
As you read, we encourage you to ask: If teams could have better, more reliable insights on user behavior, what could they do? How much faster could they work? How confident could they be about their decisions? How impactful could their prioritization be? What would be the effect on their business?

Why can't funnels answer these questions?
The short answer is that funnels are built to track user behavior that is both linear and sequential. They can only capture users who follow a prescribed set of events, and that perform those steps in a prescribed order.
The problem is that users rarely act this way!
As we all know, user behavior is rarely linear and rarely sequential. But today's digital experiences are filled with options, choices, alternate paths, and different ways to reach the same goal. For this kind of modern experience, funnels are too restricted a tool to deliver accurate information.

THE DATA
1. Unseen Detractors

DIGITAL EXPERIENCE INSIGHTS REPORT

59% of funnels miss an interaction that's associated with lower conversion.

What this means
59% of the time, teams remain unaware of a key area

Why do funnels miss so much?
As we described in the first Digital Insights Report, most teams put funnels together by assembling the steps they think users take on the way to conversion. What's the problem with this? It leaves a large amount of interactions untracked.
According to our data, for every step tracked in a funnel, almost 20 steps stay untracked.
This is a problem of what we might call "selection bias": we tend to look at the behaviors we think will be important, whether or not those are the behaviors that actually are important.

DIGITAL EXPERIENCE INSIGHTS REPORT
On the way from landing on homepage to clicking "buy," for example, there are any number of steps users take. The key dropoff event might not be the one teams are tracking -- it might be an event that's in between those steps!
If you don't have a tool that can surface these untracked intermediate steps, you might assume that the 30% dropoff is a problem with Step 3. Maybe you even put your team on fixing that problem! But what happens if the actual difficult step is one of the many intermediate steps? What if that's the key dropoff event? The crux: because traditional funnels rely on teams to choose the events they think are important, they lead teams to miss the events that are important for improvement.
© 2022 Heap Inc. | heap.io 6

THE DATA
2. Miscalculation

DIGITAL EXPERIENCE INSIGHTS REPORT

23%
of funnels miscount converting users as drop-offs, because those users skip steps.

What this means
23% of funnels deliver misleading conversion numbers. These skewed numbers can easily push teams to prioritize the wrong things.
Even worse, with traditional funnels, there's no way to know how many users are skipping steps, which steps they're skipping, and what the relative conversion rates are. (Really!)

Why do funnels miscount conversion?
As we described above, funnels require behavior to be linear and sequential. What exactly does this mean?
One way to think of it is that funnels work according to an AND logic. Given a four-step user flow, for example, a funnel analysis first captures the number of people who do Step 1, then the number of people who have done Step 1 AND Step 2, and so on. The 150 users at the end represents the number of people who did Step 1, AND Step 2, AND Step 3, AND THEN Step 4.

But let's say that it's possible for users to go directly from Step 2 to Step 4: to convert without first doing Step 3. How will these users show up in your funnel?

DIGITAL EXPERIENCE INSIGHTS REPORT

Let's see. First, the user completes Step 1. Great! Then the user completes Step 2. Also good. But then ... the funnel is waiting for the user to complete Step 3. And the user never does Step 3!
Because funnels work on AND logic. They'll count users who get to Step 4 only if those users have also done Steps 1, 2, and 3 -- in that order. If a user doesn't do any of those steps first, the funnel will count them as having dropped off -- having completed Step 2 but not moved forward in the funnel.

What this means is that users who DO convert but who skip steps along the way will be counted as NON-CONVERTING. How many are there? We don't know! But let's imagine some numbers, just to see.

DIGITAL EXPERIENCE INSIGHTS REPORT

Let's imagine you see the numbers on the left above. Looks like your overall conversion rate is 5%, with the biggest drop between Step 2 and Step 3. Wow -- if you could increase the conversion between Step 2 and Step 3, you could probably make a big impact on the business.
But, what if users aren't actually dropping off, but are just skipping Step 3? What if there are actually 300 users who do just that?
Now you get a very different picture of user behavior. What you thought was aterriblyperforming funnel (5% conversion rate) is actually a SUPER-performing funnel -- 35% conversion!

AND ... what you saw as the greatest moment of friction -- moving from Step 2 to Step 3 -- isn't actually friction at all. It's just that Step 3 is optional, so lots of users skip it.
What's the result? Instead of focusing on reducing friction at Step 3, you might prioritize something else. After all, this funnel is working great!
But note, in 23% of funnels, you can't see that users skip steps. What do funnels conceal? Turns out: A LOT.

THE DATA
3. Invisible Paths

DIGITAL EXPERIENCE INSIGHTS REPORT

63% of funnels contain an alternative pathto conversion -- that isn't tracked

What this means
63% of funnels miss the path users actually take to conversion. This leads to profound misunderstandings about what users actually do, and what improvements should be prioritized.

Why can't funnels compare alternate paths?
Above we talked about selection bias. Now let's examine another limit of funnels: their ability to compare alternate paths.
Let's say you're running an ecommerce site. Your site has two basic ways to get from your homepage to "add an item" to your cart: you can search, or you can browse by category.

As the site owner, you're interested in two questions:
1. How many people use each option? 2. Which option converts better?
As a digital experience owner, how would you answer this? You might be tempted to create two funnels and compare them. Something like the example below: identical funnels, with one tracking people who search before adding to cart, and the other tracking people who browse categories before adding to cart.

How you might think you can compare paths to the same goal. Turns out this method is highly misleading.

DIGITAL EXPERIENCE INSIGHTS REPORT
Looks great! Except ... what about people who do both? Maybe someone starts by browsing through the categories in the nav, but then remember exactly what they wanted and put it in the search bar. What happens with those people -- the ones who first do categories, then search, then add to cart? Let's call them Person Y. How does Person Y show up in your analysis? Take a look: Person Y would be counted in both funnels.

On the top, Person Y satisfies the criteria for converting Funnel A: they land on the site, use the search function, then add something to cart. Does Funnel A care if they also viewed the categories? Not at all! Same thing on the bottom: Person Y lands on the site, views categories, then adds to cart -- all the criteria for converting in Funnel B.

So any Person Y -- anyone who does both search and category before adding to cart -- will count as having converted in both Funnel A and Funnel B. Which...
1. Makes it hard to know how many people did Funnel A as opposed to Funnel B, and
2. Makes it impossible to figure out the thing you really care about, which is the relative} conversion rates between Funnels A and B!

DIGITAL EXPERIENCE INSIGHTS REPORT

Plus ... we're ignoring all the people who may have done either search or category after having added to cart! Let's say there's a Person Z who gets to the site, uses the category nav to find their item, buys the item, gets sent back to the homepage, and then uses search to find something else. Where do they end up in the funnels?
Funnel B will count them as it probably should: as a converting user who viewed the category nav first. But let's also notice that they'd show up in Funnel A's count as well!
As happens in the previous example, Funnel A would count this person as a non-converting user who dropped off at Step 2! From the perspective of Funnel A, the fact that Person Z used the category nav and then bought something isn't relevant: after a user lands on the site, Funnel A will just wait until they use the search bar, and increase the count when they do.
Could you find a way to eliminate all the Person Z's from the count you get in Funnel A? It might be mathematically possible, but it would take so much work that it would simply not be worth it.

Let's review.
You start off with two simple questions:
1. How many people take each path to conversion?
2. What's the relative conversion rate between the two paths?
And it turns out that funnels let you answer neither! As in the same example, there are users who will be counted in both funnels, users who count as converted in one funnel and unconverted in the other, users who convert but are registered as unconverted. And, most importantly, there's no way to know how many there are.
That is, in addition to not being able to tell you the answers to your questions, funnels actually conceal information about users who perform actions in both paths.

Conclusion

DIGITAL EXPERIENCE INSIGHTS REPORT

As was shown, funnels -- at least as traditionally constructed -- are a tool that's both misleading and inaccurate. Because they're sequential and linear, they're not able to capture moments when users face a choice.
The truth is that modern digital experiences offer nothing but choices!
· Can you afford to be missing a significant percentage of what your customers are doing on your site?
· How much better would you prioritize if you could see all the paths users took on your site?
· What would the dollar impact to the business be to those more informed decisions, and how much more confident would your decisions be?
What if you could have a tool that delivered accurate numbers and captured what actually happens when users have a choice?

To learn more about how you can get real insight into your customer's digital journey go to heap.io

About Heap
Heap is the premier system of insight for digital experience builders. Our mission is to illuminate hidden opportunities for fast-moving digital teams to delight their customers and move the needle on key metrics. Over 8,000 businesses use Heap to increase revenue, improve conversion, accelerate decision-making, and drive business impact at scale. Visit heap.io to learn more.
© 2022 Heap Inc. | heap.io 13

References

Adobe InDesign 17.2 (Macintosh) Adobe PDF Library 16.0.7