How to Create Metrics
This article is a comprehensive guide that will help you create different types of metrics.
Background
Metrics are used to measure the success of your experiment. ABetterChoice supports seven different types of metrics, including Average, User Proportion, Ratio, Sum, User Count, Active Days Per User, and Retention. It's important to understand the differences between these types, and this article will provide you with all the information you need to get started. This guide is intended for all users.
Metric Definitions | Description |
---|---|
Average | Describe as Sum/UV or PV/UV, such as Average Revenue, Sessions per User |
User Proportion | Described as UV/UV, such as The proportion of users who clicked the Share button |
Ratio | Described as UV/UV, such as The proportion of users who clicked the Share button |
Sum | Describe as PV or Sum, such as Total Amount Paid |
Retention | Describe as UV/UV, such as fraction of the previous day's (DO) users who are active the following day (D1) |
Active Days Per User | Describe as Sum(user_day)/UV, such as Login Days Per User |
User Count | Describe as UV, such as Total Unique Views |
Metrics will be defined upon user action events table. ABC's Stats Engine will join the user action events with the experiment exposure events to compute the metric value for your experiment.
Where to Create
Go to the Metrics section in the left sidebar, select the "Metrics Catalog", and click on the Create button.
Fill in the Basic Information
● Name
Names can only contain letters, numbers, and underscores. Once the metric is created, the name can't be modified. In a project, metric names should be unique.
● Description (optional)
Describe the purpose and meaning of the metric to ensure that other users have a better understanding of the metrics and their relevance within the project.
● Owner
The metrics creator is the default owner, and additional owners can be added as needed.
Complete the configuration
● Metric Type
Available metrics types: Average, User Proportion, Ratio, Sum, User Count, Actibe Days Per Users, Retention - Unique Users and Retention - All Users.
You can make selections from the dropdown list according to your requirements. For each type, please refer to the displayed definitions and ABetterChoice.ai to learn more.
*Note: Based on the selected metrics type, the displayed content below will also vary, primarily in the Formula section. Below is an example using "Average" to illustrate.
● Select Source Table
Choose the required data source here, and the previously created tables will be displayed in the dropdown list. For more information about tables, please refer to ABetterChoice.ai. Furthermore, if necessary, you have the option to apply filtering conditions to the data source. Simply click on the icon located on the right-hand side and enter your desired criteria in the provided input box (e.g., play_time > 0).
● Field Aggregation
Choose the user level aggregated values here. Select the table on the left and choose the field on the right, along with its corresponding aggregation rule. Furthermore, if necessary, you can simply click on "+ Add Field" to add multiple fields.
○ Available aggregation rules: SUM: Calculates the sum of a user's field values. COUNT: Calculates the count of a user's fields. Custom: To implement this, select "Custom" and enter your desired criteria in the provided input box.
● Filter by Field Aggregation (optional)
By default, it is not enabled. If you enable it, you can apply additional data filtering based on aggregated values at the user level. To do this, choose the appropriate aggregate field from the previous step, along with the relational operator and the desired value to complete the filtering.
Furthermore, if necessary, you can simply click on "Filter Conditions" to add multiple filters and define the relationship between them using "AND/OR".
● Formula
Used for calculating the value of the metric.
○ Definitions for common numerator and denominator terms: (Taking aggregated values 'A' as an example) Sum (A): The sum of A Count (A>0 Users): The number of unique users who satisfy the condition A>0 Count (Participations of the Experiment): The number of unique participations of the experiment.
● Advanced
By default, the following fields are set as follows and can be modified as needed.
Expected direction: The expected direction of the representative indicator (e.g., the higher the metric value, the better). Conversion factor: For example: If the original unit of the metric is seconds and it needs to be converted to minutes, the conversion factor is 1/60.
Minimum Detectable Effect Threshold: Used for experiment's minimum required sample size estimation.
Finish Creation
After clicking the "Create" button, you will receive a successful creation notification, and the newly created metrics will be visible on the "Metrics -- Metrics Catalog" page.