Chapter 4

Retention metrics: How to properly measure retention once and for all

Retention isn’t a tip or hack, like an automated email sequence. It goes a little deeper.

“Retention is the core of your growth model and influences every other input to your model,” as Brian Balfour, CEO of Reforge and former VP of Growth at HubSpot, says.

When you improve retention, you improve the rest of your funnel. So, how do you improve retention? By measuring the right metrics.

Downloads, pageviews, and registered users are the wrong metrics. These are vanity metrics.

Let me ask you: Which metric would help you make a better business decision?

Vanity metrics are convenient. The right metrics help you discover the truth.

As Hiten Shah, Co-Founder of Kissmetrics, Quick Sprout, & Product Habits, describes “Your metrics to a certain degree should make your life uncomfortable, it should push you to work harder.”

Customer lifetime value, cost of acquisition, churn and engagement are examples actionable metrics.

Let’s dive into how we measure these metrics.

First up, customer lifetime value (LTV).

Customer value is made up of a direct relationship between the lifetime value of a customer (LTV) and the cost of acquisition (CAC).

Over time, you want to see LTV increase and CAC decrease to build a thriving, sustainable product.

There are a few ways to do that.

On one side of the equation, you can decrease costs of acquisition by getting better at marketing, sales, and promotion. Cost of acquisition is an efficiency rate that shows how good you are at transforming eyeballs into cash through campaigns.

Lifetime value, on the other hand, is made up of revenue per user (or how much it costs to serve each user), and then the length of time that user sticks with you before leaving.

The Achilles’ Heel behind LTV is churn.

Venture Capitalist, David Skok, signals churn out as perhaps the most important indicator for retention.

“The importance of a low churn rate cannot be overstated. If your churn rate is high, then it is a clear indication of a problem with customer satisfaction.”

Eric Ries’ rule of thumb for product growth comes down to a simple formula:

“Your rate of new customer acquisition has to exceed the churn rate.”

Churn, therefore, is like the antithesis of Retention. And it has the power to erode new acquisition.

Churn acts like financial leverage.

You borrow money to invest. That investment goes off well, and your return increases exponentially. But if it goes down, it decreases exponentially, too. It puts you in a worse spot than before.

Churn can have the same amplification or compressing effect on product growth. David Skok says, “if you can halve your churn rate, it will double your LTV.”

Of course, churn isn’t a one-time calculation. It’s a moving target, changing every day.

And that’s what makes accurately measuring it so difficult. There are so many different variables involved.

Your churn rate can be measured over periods of time, so that you can see changes from month to month. Or it can be applied over cohorts, so that you can dig into different rates for each customer plan type.

Contracts with automatic renewals are easy to measure.

Otherwise, you can still use the customer renewal date, or in this case, when someone initially purchased, to help define the sales cycle with a natural beginning and end.

From there, you can look into product usage to gauge customer satisfaction. Generally speaking, more usage indicates a lower potential to churn.

Take an email marketing product like MailChimp for example.

Here, “more usage” can be defined as anything from:

  • More aggregate logins during the period

  • A higher frequency of daily logins

  • More campaigns created or sent

  • More emails sent to

  • More subscribers added

  • And more.

Each can be instrumented to track individual usage and compare against the average of all customers or those in the same cohort.

Plotting these numbers over time can help you identify trends and whether customer satisfaction is increasing or decreasing over time.

Alastair Mitchell, CEO of Huddle, told David:

“I actually think this [cohort analysis] is more important than churn, for the simple fact that churn varies over the lifetime of a customer cohort, and just looking at monthly churn can be very misleading.”

Churn MRR is your churn rate against your monthly recurring revenue. It’s the MRR lost because of churn.

A simple churn percentage doesn’t always show you how churn is negatively affecting your bottom line. A few percentage points doesn’t sound too bad on the face of it.

But when you put it in dollars and cents with Churn MRR, the pain becomes all too real. It helps you stack up each component to see how new customers and revenue continue to be eroded over time.

Churn MRR also leads you to the Quick Ratio, which looks at the discrepancy between new and lost MRR.

Ideally, higher is better.

It means new or ‘added’ MRR is growing faster than the MRR you’re losing through contraction or churn.

Baremetrics breaks into down into four components of MRR:

  • New: Brand new customers

  • Expansion: Increases from existing customers through upgrades or upsells.

  • Contraction: Existing customers downgrading their plans

  • Churned: Customer cancellation or losses

InsightSquared cites an ideal benchmark of around four for rapidly expanding products. The reason is that it helps you determine the ‘acceptable’ level of churn that your new growth can sustain without flipping revenue upside down.

But that’s just a rough estimate depending highly on your own unique business model and product. So start by raising your own historical trends, first, before striving to match this external benchmark.

Average Revenue Per User (ARPU) is what it sounds like: Total MRR divided by active customers.

Once again, higher is better. But it’s not always realistic.

ARPU helps you determine which levels of investment are acceptable in places like customer service or acquisition.

Earlier, we looked at the three different kinds of product tutorial onboarding. If your ARPU is low, one-on-one customer calls are out of the question. They’re too expensive, and don’t scale well enough to support the high volume of customers needed to grow. Same goes for acquisition costs.

AdWords might convert well. However, the Cost Per Lead on a single customer is probably too expensive at scale to support a lower ARPU.

Instead, you need an acquisition strategy that lowers the cost to acquire each lead, like through content and SEO. It might take longer to ramp up than AdWords, but the difference is that it will be sustainable growth your business model can support.

Unsurprisingly, churn also affects this equation.

The types of customers you lose (big vs. small) has a direct impact on ARPU, raising or lowering it exponentially in some cases if a lot of your revenue is tied up in a few big customers.

Think about a B2B company with subscription revenue.

It takes a lot of little $10,000 annual contracts to make up the revenue of a few $100,000/year ones. Losing a whale or two can drastically change ARPU overnight.

Next, Annual Run Rate is your MRR multiplied by twelve months. It gives you a future projection based on your most current revenues, assuming everything stays the same.

Annual Run Rates provide a fast, easy benchmark. However, it can also neglect to show you the full picture of how profitable your product is (or how efficiently it’s growing).

The other big problem is that it doesn’t take into account big fluctuations. This is especially an issue in the early days, where large customer accounts can often distort reality.

The last piece to the puzzle is engagement.

Once you have your data for MRR, you can start to look for areas where users are consistently dropping off to determine engagement. This is often the missing piece of retention reporting.

Take a look at what behaviors are causing drop-offs and why.

Facebook retains 98% of users 90 days after install. Whatsapp retains 77%. And, Instagram retains 48%.

Those are pretty big numbers. If you want to survive past the 90 days, you need to “build methodical and systematic approaches for deepening engagement.”

Front used engagement metrics to grow revenue from their current customers. They figured out what their current customers loved most about their app and delivered it.

Front grew their existing customer base by 50% by delivering what their high-value customers want. Their success came from integrating customer retention with acquisition and onboarding.

Slack is another example. Just look at these before and after screenshots.


Image Source


Image Source

Slack’s product team talks about their onboarding process stating:

“Your onboarding should equip people to get something done as quickly as possible, no matter what context they have. Determine the key tasks you need a user to accomplish in the first 30 seconds of interacting with your app, and design your onboarding to help them get it done.”

Essentially, the goal is to move users from one stage to the next—making sure your long-term users to stay engaged over time. More engaged users means higher retention rates.

While not exhaustive, these metrics help you gain a full picture of your retention. You’re able to measure it from many different angles, seeing exactly how devastating a high churn rate can be to both product and revenue success.

These metrics also help you make an important connection: Product growth is directly attributed to customer success.

It almost doesn’t matter how good you are at adding new customers if you can’t keep the existing ones around. Ultimately, a high churn will undermine everything else.

The only way you’re going to keep churn low, and thus boost retention, is to build a product that users want to come back to over months and years. Not just weeks.