Ani’s Newsletter

Share this post

How Meta uses analytics to assess Product Market Fit

aniganti.substack.com

How Meta uses analytics to assess Product Market Fit

We take a look at the framework Meta uses to assess product-market fit for their products

Ani Ganti
Jan 24
2
Share this post

How Meta uses analytics to assess Product Market Fit

aniganti.substack.com

Meta has built and continues to build some of the most prolific digital products of our time - but how do they do it repeatedly? They’ve built some duds but Meta has had a pretty impressive run in the last 20 years. The company built up their data and analytics capabilities to support their philosophy to “Build Fast and Break Things”. In this edition we look at how Meta’s analytics team helps to answer the following question for products they build:

Are we building something people want?

  • Answering such an open-ended and ambiguous question is hard, even for a company like Meta

    • Meta needed a way to reliably and quantitatively evaluate product-market fit (PMF) for their products

    • They developed a PMF Playbook that identifies 3 criteria to measure PMF: stable retention, sustainable growth and deep engagement

    • I dive deep into the growth accounting framework that Meta uses to evaluate PMF for products


What is Product-Market Fit (PMF) ?

PMF is the ability of a given product to meet the needs and desires of a specific market segment. Achieving PMF requires an in-depth understanding of customer segmentation, market trends, and technology. Product teams must understand how different segments within their target market are affected by changes in the industry and what types of products they need to satisfy their individual requirements.

Thanks for reading Ani’s Newsletter! Subscribe for free to receive new posts and support my work. It really helps :)

When finding PMF, it’s important to note three things:

  1. Find your target segment — Don’t try to boil the ocean. Instead, be laser focused on finding a market segment for your product. Define the segment, try to understand it and address its needs.

  2. PMF is a moving goalpost — The value delivered by your product can change over time as the market, target segment and/or product evolves. So PMF should be a guardrail measure that should be continuously monitored.

  3. PMF is non-binary — PMF can actually be quite vague. There is no clear cut answer to what PMF “looks like” in practice. PMF is about the value being delivered by your product to your customers in its target segment. It’s not a checkbox that can be checked and forgotten.

User Activity == PMF

Measuring user activity is usually the best proxy to measure PMF. If people enjoy what you have to offer, they'll stick around. Meta uses the following tenets when trying to measure user activity:

Defining what “Active” mean for your product

When it comes to defining a user’s activity, there are some factors to consider. For e-commerce or marketplace products, it could be whether they made an online purchase (active buyer) or created an ad (active advertiser). But for other products, the answers aren’t so cut and dry; users’ engagement is determined by various metrics like logging in, interacting with specific elements of the product or spending particular amounts of time within it.

At Meta, the product team tries to define “active” as a user action that matches a product’s core value. For instance, for Instagram, “active” can be defined as someone who has consumed content or produced content (watched a reel, posted a story etc.).

It's important to be aware of how far along the customer journey each “active” action is. Actions which are earlier in the process may result in more users being active at any given time (eg. they logged in to Instagram); however, later actions tend to demonstrate a greater value gained by those customers who have stayed on their path through your product or service (eg. they posted a story).

📊 Remember, activity is a key measure of whether people find value in your product

Fun fact: Meta has switched to using the term “Active People” instead of “Active Users”

Clearly delineating the target segment(s)

There should be a specific, definable market segment that is the focus of your product. Your product should address that segment’s needs extremely well to grow and be successful. To achieve PMF, it's essential to first decide who your target market segment should be. Qualitative research can steer you towards the core problem that needs to be solved and the people it solves it for.

Note that a product can exist across multiple market segments too, especially if it’s a “marketplace” product. In such cases, the definition of “active” will be different for each segment — an active buyer is different than an active seller on FB Marketplace). Meta sees this in almost all of their flagship products:

  • Instagram Reels segments include content creators (Reels producers) and content consumers (Reels consumers)

  • FB Marketplace segments include buyers and sellers

  • Meta Quest segments include the VR game developers and the VR game players

Meta’s Framework to measuring PMF

Meta the following three key metrics to assess the value delivered by products within their markets and gauge PMF.

  1. Stable Retention — Are people coming back predictably?

  2. Sustainable Growth — Is the product able to consistently acquire, retain and resurrect users?

  3. Deep Engagement — Are people using the product? If so, how long are they using it for?

1 - Measuring stable retention with X-day Retention

Retention is a key measure of the effectiveness of your product. If customers find it valuable, they will continue to use it. Retention measures the percentage of users that are still active after a defined window of time since their first action date. Say 1000 people first tried your product on Jan 1, 2023. If 300 of those people were active on January 8, 2023 (ie. on Day 7) then your product has a day-7 retention of 30%.

Comparing your product’s retention against benchmarks can also help gauge PMF.

Lenny's Newsletter
has an amazing list of industry benchmarks that you can reference.

Lenny's Newsletter
What is good retention
Hello, and welcome to the free monthly edition of my weekly newsletter. I’m Lenny, and each week I tackle reader questions about product, growth, working with humans, and anything else that’s stressing you out at the office. If you’re not a paid subscriber, here’s what you missed this month…
Read more
3 years ago · 84 likes · 60 comments · Lenny Rachitsky

2 - Measuring sustainable growth using Growth Accounting

While growth can be a valuable indicator of PMF it's not the only metric to consider. In fact, if growth is the only focus, it can actually be detrimental to the success of your product. Prioritizing growth over user experience can lead to a product that fails to engage and retain customers in the long term. To truly achieve PMF, it's essential to balance growth and user experience. This will ensure your product not only attracts customers, but also keeps them coming back for more.

At Meta’s scale where their products are used by billions of people, the company uses growth accounting to evaluate their products’ growth trajectory and understand their sources of growth.

“We often limit top-of-funnel acquisition until we are confident that our products can sustainably add and retain new people.”

Meta assigns five growth accounting states to each person that has used the product.

  1. New — The person used the product for the first time today (user-acquired)

  2. Retained — The person was active on the product yesterday and today

  3. Churned — The person was active on the product yesterday but not today

  4. Resurrected — The person was inactive on the product yesterday but was active today

  5. Stale — The person was inactive in the product yesterday and was inactive again today

Please consider sharing this with Product and Analytics folks in your network. Thank you!

Share

Image showing the 5 growth account states for Meta products
Source: Meta’s blog

Meta uses these five states to evaluate key metrics about product growth. For example:

Net User Growth = New users + Resurrected Users - Churned Users

Active Users = New users + Resurrected Users + Retained Users

By using growth accounting, Meta is able to continuously evaluate if the product is growing and if it is, it’s an indicator of PMF.

3 - Measuring deep engagement using metrics like ARPU, L28 and Time in-app

Using engagement as a key measure to evaluate PMF should not come as a surprise. Engagement gives a sense of how deeply people are interacting with the product and deriving value from it. Meta uses a variety of engagement metrics like:

  • Average Revenue Per User — The avg. spend per day/week/month

  • L28 — The number of days in the last 28 days that a person used your product

  • Time in-app — The average duration a person spends in your product


Meta's framework of measuring a product's value to market
Meta’s PMF Playbook

To understand the value of a product in the market, we need to consider three key factors: retention, growth and engagement. A product that has a high degree of PMF will retain customers, sustainably acquire new ones, and be used frequently and for extended periods of time. These three factors together provide a comprehensive picture of a product's value in the market and ensure that a high-quality product is being built.

For more details, read the full blog post by Meta’s Analytics team here.

Thanks for reading Ani’s Newsletter! Subscribe for free to receive new posts and support my work.

Share this post

How Meta uses analytics to assess Product Market Fit

aniganti.substack.com
Comments
TopNewCommunity

No posts

Ready for more?

© 2023 Ani Ganti
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing