Introduction to machine learning · Question 47 of 60
A model has been running in production for months; its precision is gradually falling below the acceptance criterion. What should happen next?
2 answers are correct. Hard
Show answer
Answer
- C Identify the drift through regular evaluation against the acceptance criteria and train a new model with new data and/or hyperparameters
- D Compare the new model with the existing one using a form of A/B testing
Explanation
A model can gradually get worse during operation. That is why you regularly measure whether it still meets its acceptance criteria. If it needs replacing, you retrain, with fresh data, different hyperparameters or both, and pit the new model against the old one in a kind of A/B test.