Deployment testing · Question 48 of 60
An online retailer wants to introduce a new recommendation model gradually and detect regressions early, before all users receive it. Which deployment test fits, and how does it work?
2 answers are correct. Hard
Show answer
Answer
- C Canary testing
- D The model is first released to a small subset of traffic, and latency, accuracy and error rates are monitored in real time
Explanation
In canary testing, only a small share of users gets the new model at first, for example 5%. Latency, accuracy and error rate are observed live, and only if nothing gets worse does everyone receive the update.