Experiments

What is "Experiments"?

“Experiments" is the page where you can oversee and conduct randomized experiments. It facilitates easy analysis of your features' impact with scientific rigor.

When to use "Experiments"?

"Experiments" is ideal for running A/B or A/B/n experiments or when you want to run multiple mutually exclusive experiments in parallel.

What is A/B Testing?

A/B testing originated from the double-blind tests in biomedicine. In double-blind tests, patients are randomly divided into two groups and given a placebo and a test drug, respectively, without their knowledge. After a period of observation, the experimenters compare whether there is a difference in the changes in the patients' conditions between the two groups, and thus judge whether the test drug is effective. In real business, we also leverage the A/B testing concept. Suppose there are two solutions, A and B, in the current product cycle (for example, solution A uses a red banner, and solution B uses a blue banner). You can test the impacts of solutions A and B on randomly sampled users, collect the performance data, and then, based on the result, implement the winner solution to your product.

Why we need A/B Testing?

【Real Online Feedback】A/B testing is a great tool to measure the performance of different versions of an experience. It captures the causal relationship between the solution variations and their effects with statistical validity. At the same time, it can accurately and effectively quantify the incremental impact brought by the solutions. 【Low Cost, Low Risk】Large tech companies launched various optimization strategies (including UI design, algorithm optimization, etc.) every quarter each year. A/B testing aids these businesses in boosting product profits by identifying the best stategies with low-cost and low-risk.