Generative AI foundations · Question 6 of 60

What is a token in a language model?

1 answer is correct. Easy Created with AI

Show answer

Answer

  • A A small unit of text, such as part of a word

Explanation

Tokenization splits text into smaller units. A token can be a single character, part of a word or a whole word. An API access key is colloquially also called a token, but that is not what is meant here.

Certifications & exams

Which exam comes next?

You can already practise for 51 exams, and 16 are up for a vote. See all certifications with their official information, cast your vote or suggest your exam.

See all certifications