Generative AI foundations · Question 51 of 60

A team compares the token count of test reports of equal length in German and English. Which statements are correct?

2 answers are correct. Hard Created with AI

Show answer

Answer

  • B The token count determines how much text fits into the context window
  • C Texts of equal length can produce different numbers of tokens

Explanation

How a text is split depends on language, word formation and the tokenizer; long compounds often break into several tokens. Since the context window is measured in tokens, counting before large inputs pays off.

Certifications & exams

Which exam comes next?

You can already practise for 51 exams, and 16 are up for a vote. See all certifications with their official information, cast your vote or suggest your exam.

See all certifications