Skip to content

Commit

Permalink
Docs: Update viewing experiment results (#10433)
Browse files Browse the repository at this point in the history
  • Loading branch information
ivanagas authored Jan 22, 2025
1 parent 354fc21 commit 106e5bf
Showing 1 changed file with 14 additions and 21 deletions.
35 changes: 14 additions & 21 deletions contents/docs/experiments/testing-and-launching.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -38,40 +38,33 @@ The best way to do this is **adding an optional override** to your [release cond

Once you test it works, you're ready to launch your experiment.

> **Note:** The feature flag is activated only when you launch the experiment, or if you've manually checked the "Enable feature flag" box.
> **Note:** While the [PostHog toolbar](/docs/toolbar) enables you to toggle feature flags on and off, this only works for active feature flags and won't work for your experiment feature flag while it is still in draft mode.
> **Notes:**
> - The feature flag is activated only when you launch the experiment, or if you've manually checked the "Enable feature flag" box.
> - While the [PostHog toolbar](/docs/toolbar) enables you to toggle feature flags on and off, this only works for active feature flags and won't work for your experiment feature flag while it is still in draft mode.
## Viewing experiment results

While the experiment is running, you can see results on the page:
While the experiment is running, you can see results on the details page:

<ProductScreenshot
imageLight = "https://res.cloudinary.com/dmukukwp6/image/upload/Clean_Shot_2024_08_16_at_16_14_33_2x_cd19cb7043.png"
imageDark = "https://res.cloudinary.com/dmukukwp6/image/upload/Clean_Shot_2024_08_16_at_16_14_14_2x_5f922eac50.png"
imageLight = "https://res.cloudinary.com/dmukukwp6/image/upload/Clean_Shot_2025_01_22_at_10_05_11_2f5017a50c.png"
imageDark = "https://res.cloudinary.com/dmukukwp6/image/upload/Clean_Shot_2025_01_22_at_10_05_31_0e81dd9c8e.png"
alt = "Viewing experiment results"
classes = "rounded"
/>

This page shows its status, whether draft, running, or complete as well as whether it has reached significance or not. It will also provide advice on how to interpret the results.

Beyond this, you can see a summary of the experiment including goals, variants, data collection, and more.

Sometimes, in the beginning of an experiment, results can be skewed to one side while enough data is still being gathered. While peeking at preliminary results is not a problem, making quick decisions based on them is [problematic](/blog/ab-testing-mistakes#3-conducting-an-experiment-without-a-predetermined-duration). We advise you to wait until your experiment has had a chance to gather enough data.

### Advanced results metrics

- **Mean:** For trends, the number who completed the goal divided by the number who were exposed.

- **Delta %:** The percentage change between the control and test variants.

- **Credible interval:** Represents a range within which we believe the true difference between the test variant and the control means lies, with 95% probability. In this context, the interval is expressed as a percentage change, indicating how much higher or lower the mean of the test variant could be compared to the control, based on the observed data and our prior beliefs.
This page shows:

- **Win probability:** The probability that variant is the best performing one.
- Status, whether draft, running, or complete as well as whether it has reached significance or not.
- Data collection progress based on an estimated target number of participants and [minimum detectable effect](/docs/experiments/sample-size-running-time#minimum-detectable-effect-mde).
- Win probability for each variant.
- [Statistical significance](/docs/experiments/statistics) for the results.
- Variants, their release conditions and distribution.
- [Primary and secondary metrics](/docs/experiments/metrics).

## Ending an experiment

When you're ready to end an experiment, you can click the **Ship a variant** button on the experiment page to roll out a variant and stop the experiment. _(Note: This button only appears when the experiment has reached statistical significance.)_
After you've analyzed your experiment metrics and you're ready to end your experiment, you can click the **Ship a variant** button on the experiment page to roll out a variant and stop the experiment. _This button only appears when the experiment has reached statistical significance._

![Ship a variant](https://res.cloudinary.com/dmukukwp6/image/upload/Screenshot_2024_08_23_at_13_52_25_6c1cf85153.png)

Expand Down

0 comments on commit 106e5bf

Please sign in to comment.