Skip to content

Commit

Permalink
Wordsmith "Supported metric types" (#10421)
Browse files Browse the repository at this point in the history
  • Loading branch information
danielbachhuber authored Jan 22, 2025
1 parent 87f2353 commit a5912fb
Showing 1 changed file with 16 additions and 9 deletions.
25 changes: 16 additions & 9 deletions contents/docs/experiments/statistics.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -54,14 +54,21 @@ With a 100% win probability, you can confidently declare the test variant as the

## Supported metric types

Experiments support a few different types of metrics, and each metric type uses a model appropriate to the shape of its data.

For example, funnel conversions are always between 0% and 100%, pageview counts can be any positive number (0, 50, 280), and property values can vary widely and tend to be right-skewed.

The following explain how Bayesian statistics is applied to each type of metric:

- [Beta model for funnel metrics](/docs/experiments/funnels-statistics) to analyze conversion rates through multi-step funnels.
- [Gamma-poisson model for trends metrics with count-based data](/docs/experiments/trends-count-statistics) like pageviews or interaction events.
- [Lognormal model with a normal-inverse-gamma prior for trends metrics with property values](/docs/experiments/trends-property-value-statistics) like revenue.
Experiments support three different types of [metrics](/docs/experiments/metrics), each with a statistical approach appropriate to the shape of its data:

1. **Funnel metrics** (like conversion rates)
- Data is always between 0% and 100%.
- Example: Percentage of users who complete a purchase.
- Uses a [Beta model](/docs/experiments/funnels-statistics).

2. **Count-based trend metrics** (like event totals)
- Data is any non-negative whole number (0, 1, 2, etc.).
- Examples: Number of pageviews, button clicks, or form submissions.
- Uses a [Gamma-Poisson model](/docs/experiments/trends-count-statistics).

3. **Value-based trend metrics** (like monetary amounts)
- Data can be any number and often has a long tail of high values.
- Examples: Revenue per user, time spent on page.
- Uses a [Lognormal model](/docs/experiments/trends-continuous-statistics).

If your experiment was created prior to January 2025, it is [evaluated using the legacy methodology](/docs/experiments/legacy-methodology).

0 comments on commit a5912fb

Please sign in to comment.