a canary in a carge to symbolize a Canary Deployment

Canary deployment, Blue/Green deployment and A/B testing compared

Overview

Canary deployment, Blue/Green deployment and A/B testing are methods to control risk. SageMaker Endpoints support these methods.

Canary deployment

A Canary release is a very risk averse deployment strategy. It involves directing a small proportion of the live traffic to the new production variant and checking that everything works as expected. The proportion of live traffic is gradually increased until all the traffic is being directed to the new production variant at which point the previous version can be removed. If any issues are identified live traffic can be switched back to the original production variant.

Blue/Green deployment

Blue Green deployments have two phases. In the first Phase the new variant is deployed in an identical environment to the production variant. It is then fed synthetic data and monitoring metrics are checked. In the second phase live traffic is switched to the new variant and the metrics are compared with those produced by the current production variant. If a problem is identified all live traffic is switched back to the production variant, otherwise the new variant becomes the new production variant and the old one is removed.

A/B testing

In Machine Learning A/B testing is used to compare the performance of a new model variant with the current one. In SageMaker the proportion of traffic split between the two model variants is configurable using the variant weight. Using this feature more than two variants could be tested at the same time if desired.

Comparison

DescriptionCanaryBlue / GreenA / B
Strategy typedeploymentdeploymenttesting, experimentation
Locationproductionproduction (two environments)production
Costcheap (one environment)expensive (two environments)cheap (one environment)
Riskvery low risk because only a small % deployedlow risk because because the deployment can be rapidly revertedsmall risk of breaking a production service
System typesmission critical systemsmission critical systemsany system
Implementation complexityComplexSimpleComplex
This is a comparison of Canary, Blue/Green deployments and A/B testing.

Credits

Photo by Kaikara Dharma on Unsplash

Whizlab’s AWS Certified Machine Learning Specialty practice exams

Practice Exams with 271 questions, Video Lectures and Hands-on Labs from Whizlabs

Whizlab’s AWS Certified Machine Learning Specialty Practice tests are designed by experts to simulate the real exam scenario. The questions are based on the exam syllabus outlined by official documentation. These practice tests are provided to the candidates to gain more confidence in exam preparation and self-evaluate them against the exam content.

Practice test content

  • Free Practice test – 15 questions
  • Practice test 1 – 65 questions
  • Practice test 2 – 65 questions
  • Practice test 3 – 65 questions
Whizlabs AWS certified machine learning course with a robot hand

Section test content

  • Core ML Concepts – 10 questions
  • Data Engineering – 11 questions
  • Exploratory Data Analysis – 13 questions
  • Modeling – 15 questions
  • Machine Learning Implementation and Operations – 12 questions

Similar Posts