Training foundation models · Question 1 of 12
What happens during the pre-training of a foundation model?
1 answer is correct. Easy
Show answer
Answer
- C It learns general patterns from huge data
Explanation
During pre-training the model processes huge, mostly unlabelled datasets and learns language, knowledge and patterns, often by predicting the next token. This is the most expensive step in the lifecycle and the basis for all later customisation.