• No products in the cart.

A/B testing, also known as split tests, is a controlled experiment with two variants, A and B in Web Analytics where two or more variants of a page are shown to users at random, and statistical analysis is used to determine which variation performs better for a given conversion goal.

The [course_title] course teaches you how to choose and characterise metrics to evaluate your experiments. You will learn to design experiment with enough statistical power, and then analyse the results and draw valid conclusions.


This course does not involve any written exams. Students need to answer 5 assignment questions to complete the course, the answers will be in the form of written work in pdf or word. Students can write the answers in their own time. Each answer needs to be 200 words (1 Page). Once the answers are submitted, the tutor will check and assess the work.


Edukite courses are free to study. To successfully complete a course you must submit all the assignment of the course as part of the assessment. Upon successful completion of a course, you can choose to make your achievement formal by obtaining your Certificate at a cost of £49.

Having an Official Edukite Certification is a great way to celebrate and share your success. You can:

  • Add the certificate to your CV or resume and brighten up your career
  • Show it to prove your success



Course Credit: Google

Course Curriculum

Module 01
Introduction to Course 00:02:00
Course Format 00:01:00
Intro to A/B Testing 00:04:00
What You Can and Can’t Do with A/B Tests 00:01:00
More A/B Testing Possibilities 00:01:00
Other Techniques 00:02:00
Overview of Business Example 00:02:00
Metric Choice 00:03:00
Estimating Click-Through-Probability 00:01:00
Repeating the Experiment 00:01:00
Which Distribution? 00:01:00
Binomial Distribution 00:04:00
Confidence Intervals 00:01:00
Calculating a Confidence Interval 00:03:00
Establishing Statistical Significance 00:01:00
Null and Alternative Hypothesis 00:03:00
Comparing Two Samples 00:01:00
Pooled Standard Error 00:01:00
Practical, or Substantive, Significance 00:02:00
Size vs. Power Trade-Off 00:01:00
How Page Views Affect Sensitivity 00:03:00
Calculating Number of Pages Views Needed 00:01:00
How Does Number of Page Views Vary? 00:01:00
Calculating Results 00:01:00
Confidence Interval Case Breakdown 00:01:00
Making Decisions about Uncertain Data 00:01:00
Conclusion 00:01:00
Module 02
Introduction 00:02:00
Tuskegee and Milgram Experiments 00:02:00
Facebook Experiment 00:02:00
Assessing Risk 00:01:00
Assessing Data Sensitivity 00:02:00
Questions and Consent 00:02:00
Which Tests Need Further Review? 00:01:00
Provided Information 00:01:00
Internal Training 00:01:00
Conclusion 00:01:00
Module 03
Lesson Introduction 00:01:00
Metric Definition Overview 00:03:00
Metric Definition Overview, Part 2 00:04:00
High Level Metrics: Customer Funnel 00:01:00
Refining the Customer Funnel 00:04:00
Choosing Metrics 00:02:00
Difficult Metrics 00:01:00
Defining Metrics: Other Techniques 00:03:00
Other Techniques, Part 2 00:05:00
Techniques to Gather Additional Data 00:03:00
Other Techniques: Example 00:05:00
Metric Definition and Data Capture 00:04:00
Metric Definition: Example 00:05:00
Filtering and Segmenting 00:04:00
Filtering and Segmenting: Example 00:04:00
Summary Metrics 00:04:00
Summary Metrics: Example 00:03:00
Sensitivity and Robustness 00:03:00
Sensitivity and Robustness: Example 00:04:00
Absolute or Relative Difference? 00:02:00
Variability 00:02:00
Variability: Example 00:06:00
Nonparametric Answers 00:02:00
Empirical Variability 00:04:00
Empirical Variability: Sanity Checking 00:04:00
Empirical Confidence Intervals 00:03:00
Empirical Variability: Bootstrapping 00:02:00
Variability Summary 00:01:00
Lessons Learned 00:05:00
Lessons Learned, Part 2 00:04:00
Lesson Conclusion 00:01:00
Module 04
Introduction 00:01:00
Unit of Diversion Overview 00:02:00
Unit of Diversion: Example 00:05:00
Consistency of Diversion 00:03:00
Consistency of Diversion: Example 00:01:00
Ethical Considerations for Diversion 00:01:00
Ethical Considerations: Example 00:01:00
Unit of Analysis vs. Unit of Diversion 00:02:00
Unit of Analysis vs. Diversion: Example 00:02:00
Inter- vs. Intra-User Experiments 00:02:00
Target Population 00:04:00
Target Population: Example 00:02:00
Population vs. Cohort 00:02:00
Population vs. Cohort: Example 00:02:00
Experiment Design and Sizing: Overview 00:01:00
Sizing: Overview 00:02:00
Sizing: Example 00:02:00
How to Decrease Experiment Size 00:02:00
Sizing Triggering 00:02:00
Duration vs. Exposure 00:03:00
Duration vs. Exposure: Example 00:01:00
When to Limit Exposure 00:01:00
Learning Effects 00:04:00
Lessons Learned 00:03:00
Conclusion 00:01:00
Module 05
Introduction 00:01:00
Sanity Checks 00:02:00
Choosing Invariants 00:01:00
Choosing Invariants, Part 2 00:01:00
Checking Invariants 00:02:00
Checking Invariants, Part 2 00:04:00
Sanity Checking: Wrapup 00:03:00
Single Metric: Introduction 00:02:00
Single Metric: Example 00:06:00
Single Metric: Gotchas 00:02:00
Gotchas: Simpson’s Paradox 00:02:00
Multiple Metrics: Introduction 00:01:00
Multiple Metrics: Example 00:03:00
Multiple Metrics: Example 2 00:03:00
Multiple Metrics: Example 3 00:03:00
Analyzing Multiple Metrics 00:04:00
Drawing Conclusions 00:03:00
Gotchas: Changes Over Time 00:03:00
Lessons Learned 005 00:02:00
Course Conclusion 00:01:00
Submit Your Assignment 00:00:00
Certification 00:00:00

Course Reviews


9 ratings
  • 5 stars0
  • 4 stars0
  • 3 stars0
  • 2 stars0
  • 1 stars0

No Reviews found for this course.

©2021 Edukite. All Rights Resereved
Edukite is A Part Of Ebrahim College, Charity Commission
Reg No 110841