Experiments #

Experiments enable you to deliver feature configurations to audiences, allowing you to test the effectiveness and success of a particular new feature, design, configuration, etc. on a given audience.

These differ from feature flags in that experiments are only for a fixed duration, are scoped to a particular audience, and are measurement-oriented - testing your assumptions about what change a new feature will drive in a given metric.

Use experiments in designing your game to work iteratively and make data-driven decisions. Successful experiments can then be converted to permanent feature flags for your players.

Creating experiments #

Click the + New Experiment button to open the Create Experiment modal, then enter the following details:

• Name: Enter a unique name for this experiment
• Description: Provide a description of the experiment

• Audiences: The audiences to participate in the experiment
• Hold Outs: The audiences, if any, to exclude from the experiment
• Goal Metrics: The desired metric to measure based on the experiment
• Monitor Metrics: Additional metrics to monitor during this experiment

Experiment details #

Select an experiment from the list to view its details page. This page contains three tabs: Details, Variants, and Phases.

Details #

View and edit the experiment’s details, including its name, description, audiences, and metrics.

Variants #

View, edit, and create new variants for this experiment.

Phases #

Create new experiment phases and view the experiment’s current and/or previous phases.

Phases are used to control the variants that are active during a given time period for a given audience split. At the conclusion of the phase, you can view the changes in the goal and monitor metrics for the experiment.