Generative AI foundations · Question 27 of 60

Which statements about tokens are correct?

2 answers are correct. Medium Created with AI

Show answer

Answer

  • B A token can be a single character
  • C A token can be part of a word

Explanation

Tokens range from single characters through parts of words to whole words. Punctuation also becomes tokens. That is why the token count cannot simply be read from the word count.

Certifications & exams

Which exam comes next?

You can already practise for 51 exams, and 16 are up for a vote. See all certifications with their official information, cast your vote or suggest your exam.

See all certifications