From a5912fb9afc5fa0d4718bc9f3aef5b6ce57d08b7 Mon Sep 17 00:00:00 2001 From: Daniel Bachhuber Date: Wed, 22 Jan 2025 11:49:32 -0800 Subject: [PATCH] Wordsmith "Supported metric types" (#10421) --- contents/docs/experiments/statistics.mdx | 25 +++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/contents/docs/experiments/statistics.mdx b/contents/docs/experiments/statistics.mdx index 9135fdaef06e..b3f87644ed27 100644 --- a/contents/docs/experiments/statistics.mdx +++ b/contents/docs/experiments/statistics.mdx @@ -54,14 +54,21 @@ With a 100% win probability, you can confidently declare the test variant as the ## Supported metric types -Experiments support a few different types of metrics, and each metric type uses a model appropriate to the shape of its data. - -For example, funnel conversions are always between 0% and 100%, pageview counts can be any positive number (0, 50, 280), and property values can vary widely and tend to be right-skewed. - -The following explain how Bayesian statistics is applied to each type of metric: - -- [Beta model for funnel metrics](/docs/experiments/funnels-statistics) to analyze conversion rates through multi-step funnels. -- [Gamma-poisson model for trends metrics with count-based data](/docs/experiments/trends-count-statistics) like pageviews or interaction events. -- [Lognormal model with a normal-inverse-gamma prior for trends metrics with property values](/docs/experiments/trends-property-value-statistics) like revenue. +Experiments support three different types of [metrics](/docs/experiments/metrics), each with a statistical approach appropriate to the shape of its data: + +1. **Funnel metrics** (like conversion rates) + - Data is always between 0% and 100%. + - Example: Percentage of users who complete a purchase. + - Uses a [Beta model](/docs/experiments/funnels-statistics). + +2. **Count-based trend metrics** (like event totals) + - Data is any non-negative whole number (0, 1, 2, etc.). + - Examples: Number of pageviews, button clicks, or form submissions. + - Uses a [Gamma-Poisson model](/docs/experiments/trends-count-statistics). + +3. **Value-based trend metrics** (like monetary amounts) + - Data can be any number and often has a long tail of high values. + - Examples: Revenue per user, time spent on page. + - Uses a [Lognormal model](/docs/experiments/trends-continuous-statistics). If your experiment was created prior to January 2025, it is [evaluated using the legacy methodology](/docs/experiments/legacy-methodology). \ No newline at end of file