Quickstart: Run your first experiment in 6 steps

This guide is for product managers, operations, and data analysts at game studios. It takes you through your first A/B experiment end to end — from creating a project to viewing results.

Engineers should go directly to the SDK quickstart; this page focuses on the end-to-end experience for product and data roles.

Running scenario: Your puzzle game shows a popup when a player fails a level. You want to test whether adding a "watch an ad to continue" option improves day-1 retention before deciding to launch it to all users.

What you need before starting

Item	Where to confirm
Team members have registered platform accounts	See Account Registration
A project (or use the Demo project)	Home page project list
A connected data warehouse	Data Management → Warehouse Native, status shows Configured
At least one defined metric	Data Management → Metrics Catalog

Step 1: Create a project

Each game has its own independent project; experiments, Remote Config, and metrics are isolated from each other.

Scenario example: Your studio published three games — create one project per game, with each team independently managing its experiments and parameters.

1. Email support@abetterchoice.ai or request project creation through the Slack community
2. Provide the project name and team permissions list

Role	Permission scope	Who it's for
Admin	Full read/write + user permission management	Project lead
Core member	Full read/write (excluding permission management)	Experiment lead, data lead
Member	Low-risk read/write (cannot launch experiments, cannot modify layers or domains, cannot change experiment parameters while the experiment is running)	Collaborators

Step 2: Integrate the SDK for experiment assignment (optional)

The ABetterChoice SDK is responsible for computing variant assignment for each player — when your code calls the SDK, the platform returns the experiment variant and parameter values that player should see, and automatically writes an exposure record.

Scenario example: A player reaches the level failure screen. The game code calls the SDK to ask "which variant does this player belong to?" The SDK returns Treatment_A with the parameter show_ad_continue = true. The game renders the new popup with the watch-an-ad-to-continue option, and the platform simultaneously logs one exposure record.

Capabilities unlocked after integrating the SDK

Capability	Description
Dual-hash assignment	Double-layer hashing ensures uniform random grouping, avoiding SRM at the source
MAB multi-armed bandit	Dynamically adjusts traffic based on metric performance; automatically favors the better variant
Adaptive assignment (in development)	Intelligently optimizes assignment based on user characteristics during the experiment
Auto-tuning experiment (in development)	The platform automatically explores the parameter space without requiring you to enumerate candidate values

Integration steps

1. Choose the applicable SDK or HTTP API — SDK and API overview
2. Initialize the SDK with your API Key
3. Call getExperiment() at the code location where assignment is needed
4. The SDK reports exposures automatically — no additional development required

You can use the platform without integrating the SDK. If your business already has its own assignment system, you can perform assignment yourself and push the assignment results as exposure data to the warehouse. The platform can still complete analysis based on your exposure data; only SDK-dependent capabilities such as dual-hash assignment, MAB, and adaptive assignment will not be available.

Step 3: Ingest exposure data

Exposure data answers one question: which player saw which version?

Scenario example: Your puzzle game's server already writes a grouping record to BigQuery at the start of each session. You just need to map that table.

Method	When to use	How
SDK auto-report (recommended)	You have integrated the SDK in Step 2	The SDK reports automatically when getExperiment() is called — no additional configuration needed
Warehouse Native	No SDK integration; game server already has grouping logs	Data Management → Warehouse Native → connect BigQuery → map your existing table as an Assignment Table — Assignment table data ingestion guide

Step 4: Ingest behavioral data

Behavioral data drives all metric calculations — ad revenue, retention, level completion rate, and purchase conversion. ABC reads directly from your warehouse without copying data.

Scenario example: Your player_events table records events such as level start, level complete, ad view, and item purchase. After one mapping, the platform can calculate day-1 retention, ad view rate, ARPU, and more.

Fact table data ingestion guide

1. Data Management → Warehouse Native — confirm BigQuery is connected
2. Data Management → Tables — map your event table as a Fact Table

Step 5: Define metrics

Metrics are the scoreboard for your experiment — they determine which version wins.

Metrics ingestion guide

Scenario example: Define D1_Retention (day-1 retention, higher is better) as the primary metric and Ad_Watch_Rate (ad view rate, higher is better) as a secondary metric for the level failure popup experiment. 1–3 primary metrics are enough to start; you can add more at any time.

1. Data Management → Metrics Catalog → Create
2. Key fields:
   • Name: e.g. D1_Retention
   • Aggregation: Retention / User Proportion / Average / Sum
   • Data source: the Fact Table mapped in the previous step
   • Expected direction: Higher is better or Lower is better

For details, see Metrics Catalog.

Step 6: Create and run your first experiment

Create

Scenario example: You want to validate whether the "watch an ad to continue" popup improves day-1 retention. The control group keeps the original popup; the treatment group adds the ad option.

Go to Experiments → Create and fill in three sections:

1. Basic information

• Experiment ID: the unique identifier referenced in code, e.g. level_fail_popup_v2 — cannot be changed after creation
• Owner: defaults to the current account; collaborators can be added
• Layer: leave unselected for your first experiment; use layers later when multiple experiments need to be mutually exclusive

2. Hypothesis and metrics

• Hypothesis: describe the change, target audience, and expected outcome. Example: "Adding a 'watch an ad to continue' option to the level failure popup will improve day-1 retention for casual players."
• Metrics: click + Metrics and select 1 primary metric (e.g. D1_Retention)

3. Groups and traffic

• Traffic allocation: the percentage of traffic assigned to this experiment (100% is fine for your first experiment)
• Variants: defaults to Control (original popup, 50%) + Treatment_A (new popup, 50%)
• Parameters (optional): config values read by the SDK, e.g. show_ad_continue = true
• Targeting Audience (optional): limit the experiment to a specific segment (e.g. players who have failed at least once)

Click Create — the experiment enters Not Started (debug) status.

Start

Open the experiment → click Start — the status changes to In Progress and exposure accumulation begins.

View results

Open the Results tab; the top section shows two cards:

• Cumulative Exposures: cumulative exposure trend per group — confirms traffic is flowing normally
• Suggestion: the platform's recommendation based on current state (e.g. "Data collecting..." or "Consider adding a primary metric")

Below that, three analysis views:

Tab	Purpose
Basic Analysis	Metric comparison — lift, P-value, confidence interval
Explore	Drill down by dimension (country, device, player level range)
HTE Analysis	Which player segments benefit most from the change

Make a decision

Once data reaches statistical significance, click Make Decision → publish the winning version or archive the experiment.

Full recap: Create project → integrate SDK → connect BigQuery tables → define day-1 retention and ad view rate → run level_fail_popup_v2 (50/50 split) → observe +1.8% day-1 retention after 5 days (p < 0.05) → launch Treatment_A to all users.

What to do next

• Run multiple experiments without interference → Layer Experiment overview
• Target only specific players → Audiences overview
• Use Remote Config for live event updates → Remote Config overview
• Understand the statistical methods in depth → Statistics engine module
• Integrate experiments into your game code → SDK and API overview