What is Making Decisions
The make decision feature of the ABetterChoice experiment platform allows users to transform successful experimental outcomes into official product features or archive experiments that do not meet expectations after the experiment cycle ends.
Why You Need the Make Decision Feature
After the completion of the experiment cycle, efficiently and accurately transforming experimental outcomes into official product features is crucial. The design of the make decision feature aims to streamline this process, enhance the efficiency from experiment to feature shipping, and ensure that product iterations can quickly respond to market and user needs. You can use the experiment shipping feature to solidify the winning experimental group's layer parameter values to the corresponding parameter's default values of the layer where the experiment is located, allowing all users on the layer to receive this new functional parameter change. You can also use the experiment archiving feature to end experiments that do not meet expectations, and users will no longer receive this experiment.
How to Make Decisions
Shipping an Experiment
- On the experiment details page, find and click the "Make Decision" button.
- In the make decision confirmation pop-up that appears, select "Ship":
- Select the experiment group to be shipped. Click the dropdown to single-select any of the experiment groups that have not been archived.
- The system will display the involved layer parameter changes. After confirming there are no errors, click "Ship" to complete the operation:
Archiving an Experiment
- In the make decision confirmation pop-up, select "Archive".
- Confirm the archiving operation. At this point, the experiment will be archived, and users will no longer receive this experiment:
Considerations
- Impact after shipping an experiment: After an experiment group is shipped, the corresponding parameter's default value in the experiment layer will be updated to the shipped group's parameter value, and the experiment traffic will become 0%. Please note that the experiment's whitelist will become invalid.
- Experiment status change: After shipping, the corresponding status in the experiment list will change to "Shipped," and the experiment details page will add a "Shipped" status indicator:
- Experiment operation log: The experiment operation log will add a record of the experiment shipping, displaying the name of the shipped experiment group.
FAQ
Q: How can I track the long-term effects of an experiment after it is shipped? A: After shipping an experiment, it is recommended to continuously track key metrics through the platform's ROI analysis tool to assess the long-term effects of the experiment shipping.
Q: What if I shipped the wrong experiment group? A: If the wrong experiment group was shipped, you can quickly go to the layer's detail page of the experiment to modify the default value of the corresponding layer parameter to minimize the impact of the shipping error.
More Help
For more information about the relationship between parameters and layers, please visit our help documentation page:Layers and Parameters