A/B testing, also known as split tests, is a controlled experiment with two variants, A and B in Web Analytics where two or more variants of a page are shown to users at random, and statistical analysis is used to determine which variation performs better for a given conversion goal.

The [course_title] course teaches you how to choose and characterise metrics to evaluate your experiments. You will learn to design experiment with enough statistical power, and then analyse the results and draw valid conclusions.

**Assessment**

This course does not involve any written exams. Students need to answer 5 assignment questions to complete the course, the answers will be in the form of written work in pdf or word. Students can write the answers in their own time. Each answer needs to be 200 words (1 Page). Once the answers are submitted, the tutor will check and assess the work.

**Certification**

Edukite courses are free to study. To successfully complete a course you must submit all the assignment of the course as part of the assessment. Upon successful completion of a course, you can choose to make your achievement formal by obtaining your Certificate at a cost of £49.

Having an Official Edukite Certification is a great way to celebrate and share your success. You can:

- Add the certificate to your CV or resume and brighten up your career
- Show it to prove your success

Course Credit: Google

### Course Curriculum

Module 01 | |||

Introduction to Course | 00:02:00 | ||

Course Format | 00:01:00 | ||

Intro to A/B Testing | 00:04:00 | ||

What You Can and Can’t Do with A/B Tests | 00:01:00 | ||

More A/B Testing Possibilities | 00:01:00 | ||

Other Techniques | 00:02:00 | ||

Overview of Business Example | 00:02:00 | ||

Metric Choice | 00:03:00 | ||

Estimating Click-Through-Probability | 00:01:00 | ||

Repeating the Experiment | 00:01:00 | ||

Which Distribution? | 00:01:00 | ||

Binomial Distribution | 00:04:00 | ||

Confidence Intervals | 00:01:00 | ||

Calculating a Confidence Interval | 00:03:00 | ||

Establishing Statistical Significance | 00:01:00 | ||

Null and Alternative Hypothesis | 00:03:00 | ||

Comparing Two Samples | 00:01:00 | ||

Pooled Standard Error | 00:01:00 | ||

Practical, or Substantive, Significance | 00:02:00 | ||

Size vs. Power Trade-Off | 00:01:00 | ||

How Page Views Affect Sensitivity | 00:03:00 | ||

Calculating Number of Pages Views Needed | 00:01:00 | ||

How Does Number of Page Views Vary? | 00:01:00 | ||

Calculating Results | 00:01:00 | ||

Confidence Interval Case Breakdown | 00:01:00 | ||

Making Decisions about Uncertain Data | 00:01:00 | ||

Conclusion | 00:01:00 | ||

Module 02 | |||

Introduction | 00:02:00 | ||

Tuskegee and Milgram Experiments | 00:02:00 | ||

Facebook Experiment | 00:02:00 | ||

Assessing Risk | 00:01:00 | ||

Assessing Data Sensitivity | 00:02:00 | ||

Questions and Consent | 00:02:00 | ||

Which Tests Need Further Review? | 00:01:00 | ||

Provided Information | 00:01:00 | ||

Internal Training | 00:01:00 | ||

Conclusion | 00:01:00 | ||

Module 03 | |||

Lesson Introduction | 00:01:00 | ||

Metric Definition Overview | 00:03:00 | ||

Metric Definition Overview, Part 2 | 00:04:00 | ||

High Level Metrics: Customer Funnel | 00:01:00 | ||

Refining the Customer Funnel | 00:04:00 | ||

Choosing Metrics | 00:02:00 | ||

Difficult Metrics | 00:01:00 | ||

Defining Metrics: Other Techniques | 00:03:00 | ||

Other Techniques, Part 2 | 00:05:00 | ||

Techniques to Gather Additional Data | 00:03:00 | ||

Other Techniques: Example | 00:05:00 | ||

Metric Definition and Data Capture | 00:04:00 | ||

Metric Definition: Example | 00:05:00 | ||

Filtering and Segmenting | 00:04:00 | ||

Filtering and Segmenting: Example | 00:04:00 | ||

Summary Metrics | 00:04:00 | ||

Summary Metrics: Example | 00:03:00 | ||

Sensitivity and Robustness | 00:03:00 | ||

Sensitivity and Robustness: Example | 00:04:00 | ||

Absolute or Relative Difference? | 00:02:00 | ||

Variability | 00:02:00 | ||

Variability: Example | 00:06:00 | ||

Nonparametric Answers | 00:02:00 | ||

Empirical Variability | 00:04:00 | ||

Empirical Variability: Sanity Checking | 00:04:00 | ||

Empirical Confidence Intervals | 00:03:00 | ||

Empirical Variability: Bootstrapping | 00:02:00 | ||

Variability Summary | 00:01:00 | ||

Lessons Learned | 00:05:00 | ||

Lessons Learned, Part 2 | 00:04:00 | ||

Lesson Conclusion | 00:01:00 | ||

Module 04 | |||

Introduction | 00:01:00 | ||

Unit of Diversion Overview | 00:02:00 | ||

Unit of Diversion: Example | 00:05:00 | ||

Consistency of Diversion | 00:03:00 | ||

Consistency of Diversion: Example | 00:01:00 | ||

Ethical Considerations for Diversion | 00:01:00 | ||

Ethical Considerations: Example | 00:01:00 | ||

Unit of Analysis vs. Unit of Diversion | 00:02:00 | ||

Unit of Analysis vs. Diversion: Example | 00:02:00 | ||

Inter- vs. Intra-User Experiments | 00:02:00 | ||

Target Population | 00:04:00 | ||

Target Population: Example | 00:02:00 | ||

Population vs. Cohort | 00:02:00 | ||

Population vs. Cohort: Example | 00:02:00 | ||

Experiment Design and Sizing: Overview | 00:01:00 | ||

Sizing: Overview | 00:02:00 | ||

Sizing: Example | 00:02:00 | ||

How to Decrease Experiment Size | 00:02:00 | ||

Sizing Triggering | 00:02:00 | ||

Duration vs. Exposure | 00:03:00 | ||

Duration vs. Exposure: Example | 00:01:00 | ||

When to Limit Exposure | 00:01:00 | ||

Learning Effects | 00:04:00 | ||

Lessons Learned | 00:03:00 | ||

Conclusion | 00:01:00 | ||

Module 05 | |||

Introduction | 00:01:00 | ||

Sanity Checks | 00:02:00 | ||

Choosing Invariants | 00:01:00 | ||

Choosing Invariants, Part 2 | 00:01:00 | ||

Checking Invariants | 00:02:00 | ||

Checking Invariants, Part 2 | 00:04:00 | ||

Sanity Checking: Wrapup | 00:03:00 | ||

Single Metric: Introduction | 00:02:00 | ||

Single Metric: Example | 00:06:00 | ||

Single Metric: Gotchas | 00:02:00 | ||

Gotchas: Simpson’s Paradox | 00:02:00 | ||

Multiple Metrics: Introduction | 00:01:00 | ||

Multiple Metrics: Example | 00:03:00 | ||

Multiple Metrics: Example 2 | 00:03:00 | ||

Multiple Metrics: Example 3 | 00:03:00 | ||

Analyzing Multiple Metrics | 00:04:00 | ||

Drawing Conclusions | 00:03:00 | ||

Gotchas: Changes Over Time | 00:03:00 | ||

Lessons Learned 005 | 00:02:00 | ||

Course Conclusion | 00:01:00 | ||

Assessment | |||

Submit Your Assignment | 00:00:00 | ||

Certification | 00:00:00 |

### Course Reviews

No Reviews found for this course.

**523 STUDENTS ENROLLED**