Training foundation models · Question 11 of 12

A pharmaceutical company has 2 million unlabelled specialist articles and 3,000 reviewed question-answer pairs. The model should understand the specialist language and answer questions in the company format. Which sequence makes sense?

1 answer is correct. Hard

1. A First continued pre-training with the articles, then fine-tuning with the pairs
2. B Pre-training a new model with only the 3,000 pairs
3. C First fine-tuning with the articles, then continued pre-training with the pairs
4. D Only distillation without the articles and without the pairs