Experiments #
Experiments enable you to deliver feature configurations to audiences, allowing you to test the effectiveness and success of a particular new feature, design, configuration, etc. on a given audience.
These differ from feature flags in that experiments are only for a fixed duration, are scoped to a particular audience, and are measurement-oriented - testing your assumptions about what change a new feature will drive in a given metric.
Use experiments in designing your game to work iteratively and make data-driven decisions. Successful experiments can then be converted to permanent feature flags for your players.
Creating experiments #
Click the + New Experiment button to open the Create Experiment modal, then enter the following details:
- Name: Enter a unique name for this experiment
- Description: Provide a description of the experiment
- Audiences: The audiences to participate in the experiment
- Hold Outs: The audiences, if any, to exclude from the experiment
- Goal Metrics: The desired metric to measure based on the experiment
- Monitor Metrics: Additional metrics to monitor during this experiment
Experiment details #
Select an experiment from the list to view its details page. This page contains three tabs: Details, Variants, and Phases.
Details #
View and edit the experiment’s details, including its name, description, audiences, and metrics.
Variants #
View, edit, and create new variants for this experiment.
Phases #
Create new experiment phases and view the experiment’s current and/or previous phases.
Phases are used to control the variants that are active during a given time period for a given audience split. At the conclusion of the phase, you can view the changes in the goal and monitor metrics for the experiment.