How Meta uses analytics to assess Product Market Fit
Discover more from Ani’s Newsletter
How Meta uses analytics to assess Product Market Fit
We take a look at the framework Meta uses to assess product-market fit for their products
Meta has built and continues to build some of the most prolific digital products of our time - but how do they do it repeatedly? They’ve built some duds but Meta has had a pretty impressive run in the last 20 years. The company built up their data and analytics capabilities to support their philosophy to “Build Fast and Break Things”. In this edition we look at how Meta’s analytics team helps to answer the following question for products they build:
Are we building something people want?
Answering such an open-ended and ambiguous question is hard, even for a company like Meta
Meta needed a way to reliably and quantitatively evaluate product-market fit (PMF) for their products
They developed a PMF Playbook that identifies 3 criteria to measure PMF: stable retention, sustainable growth and deep engagement
I dive deep into the growth accounting framework that Meta uses to evaluate PMF for products
What is Product-Market Fit (PMF) ?
PMF is the ability of a given product to meet the needs and desires of a specific market segment. Achieving PMF requires an in-depth understanding of customer segmentation, market trends, and technology. Product teams must understand how different segments within their target market are affected by changes in the industry and what types of products they need to satisfy their individual requirements.
When finding PMF, it’s important to note three things:
Find your target segment — Don’t try to boil the ocean. Instead, be laser focused on finding a market segment for your product. Define the segment, try to understand it and address its needs.
PMF is a moving goalpost — The value delivered by your product can change over time as the market, target segment and/or product evolves. So PMF should be a guardrail measure that should be continuously monitored.
PMF is non-binary — PMF can actually be quite vague. There is no clear cut answer to what PMF “looks like” in practice. PMF is about the value being delivered by your product to your customers in its target segment. It’s not a checkbox that can be checked and forgotten.
User Activity == PMF
Measuring user activity is usually the best proxy to measure PMF. If people enjoy what you have to offer, they'll stick around. Meta uses the following tenets when trying to measure user activity:
Defining what “Active” mean for your product
When it comes to defining a user’s activity, there are some factors to consider. For e-commerce or marketplace products, it could be whether they made an online purchase (active buyer) or created an ad (active advertiser). But for other products, the answers aren’t so cut and dry; users’ engagement is determined by various metrics like logging in, interacting with specific elements of the product or spending particular amounts of time within it.
At Meta, the product team tries to define “active” as a user action that matches a product’s core value. For instance, for Instagram, “active” can be defined as someone who has consumed content or produced content (watched a reel, posted a story etc.).
It's important to be aware of how far along the customer journey each “active” action is. Actions which are earlier in the process may result in more users being active at any given time (eg. they logged in to Instagram); however, later actions tend to demonstrate a greater value gained by those customers who have stayed on their path through your product or service (eg. they posted a story).
📊 Remember, activity is a key measure of whether people find value in your product
Fun fact: Meta has switched to using the term “Active People” instead of “Active Users”
Clearly delineating the target segment(s)
There should be a specific, definable market segment that is the focus of your product. Your product should address that segment’s needs extremely well to grow and be successful. To achieve PMF, it's essential to first decide who your target market segment should be. Qualitative research can steer you towards the core problem that needs to be solved and the people it solves it for.
Note that a product can exist across multiple market segments too, especially if it’s a “marketplace” product. In such cases, the definition of “active” will be different for each segment — an active buyer is different than an active seller on FB Marketplace). Meta sees this in almost all of their flagship products:
Instagram Reels segments include content creators (Reels producers) and content consumers (Reels consumers)
FB Marketplace segments include buyers and sellers
Meta Quest segments include the VR game developers and the VR game players
Meta’s Framework to measuring PMF
Meta the following three key metrics to assess the value delivered by products within their markets and gauge PMF.
Stable Retention — Are people coming back predictably?
Sustainable Growth — Is the product able to consistently acquire, retain and resurrect users?
Deep Engagement — Are people using the product? If so, how long are they using it for?
1 - Measuring stable retention with X-day Retention
Retention is a key measure of the effectiveness of your product. If customers find it valuable, they will continue to use it. Retention measures the percentage of users that are still active after a defined window of time since their first action date. Say 1000 people first tried your product on Jan 1, 2023. If 300 of those people were active on January 8, 2023 (ie. on Day 7) then your product has a day-7 retention of 30%.
Comparing your product’s retention against benchmarks can also help gauge PMF.
has an amazing list of industry benchmarks that you can reference.
2 - Measuring sustainable growth using Growth Accounting
While growth can be a valuable indicator of PMF it's not the only metric to consider. In fact, if growth is the only focus, it can actually be detrimental to the success of your product. Prioritizing growth over user experience can lead to a product that fails to engage and retain customers in the long term. To truly achieve PMF, it's essential to balance growth and user experience. This will ensure your product not only attracts customers, but also keeps them coming back for more.
At Meta’s scale where their products are used by billions of people, the company uses growth accounting to evaluate their products’ growth trajectory and understand their sources of growth.
“We often limit top-of-funnel acquisition until we are confident that our products can sustainably add and retain new people.”
Meta assigns five growth accounting states to each person that has used the product.
New — The person used the product for the first time today (user-acquired)
Retained — The person was active on the product yesterday and today
Churned — The person was active on the product yesterday but not today
Resurrected — The person was inactive on the product yesterday but was active today
Stale — The person was inactive in the product yesterday and was inactive again today
Meta uses these five states to evaluate key metrics about product growth. For example:
Net User Growth = New users + Resurrected Users - Churned Users
Active Users = New users + Resurrected Users + Retained Users
By using growth accounting, Meta is able to continuously evaluate if the product is growing and if it is, it’s an indicator of PMF.
3 - Measuring deep engagement using metrics like ARPU, L28 and Time in-app
Using engagement as a key measure to evaluate PMF should not come as a surprise. Engagement gives a sense of how deeply people are interacting with the product and deriving value from it. Meta uses a variety of engagement metrics like:
Average Revenue Per User — The avg. spend per day/week/month
L28 — The number of days in the last 28 days that a person used your product
Time in-app — The average duration a person spends in your product
To understand the value of a product in the market, we need to consider three key factors: retention, growth and engagement. A product that has a high degree of PMF will retain customers, sustainably acquire new ones, and be used frequently and for extended periods of time. These three factors together provide a comprehensive picture of a product's value in the market and ensure that a high-quality product is being built.
For more details, read the full blog post by Meta’s Analytics team here.