ISTQB CT-GenAI
Generative AI Fundamentals for Testing
Easy
Generative AI foundations
- Which type of AI represents knowledge through symbols and explicitly written logical rules?
- In which type of AI do people select the features the model learns from before training?
- What is deep learning based on?
- What sets generative AI apart from the other types of AI?
- What is the main advantage of generative AI for test tasks compared with classical machine learning?
- What is a token in a language model?
- What are embeddings?
- What does the context window of an LLM refer to?
- What is a small language model (SLM)?
- Which neural network architecture are today's LLMs based on?
- What does an LLM compute step by step during inference?
- Why can the same prompt give a different answer when it is sent a second time?
- What is a foundation LLM?
- What is an instruction-tuned LLM further trained with?
- What are reasoning LLMs especially designed for?
- What can a multimodal LLM do that a text-only LLM cannot?
- What do vision-language models combine?
Generative AI in software testing
Medium
Generative AI foundations
- A team trains a model on 40,000 historical defect reports classified by hand, using preselected features such as module and severity. Which type of AI is this?
- Which activities often remain a human task even with deep learning?
- What applies to the embeddings of two tokens with a similar meaning, such as "crash" and "abort"?
- What are the consequences of a larger context window?
- An LLM suggests an expected value for a test case that sounds very convincing. Which assessment is correct?
- A tester wants to pass a 300-page system specification to an LLM. What should she clarify first?
- Which statements about tokens are correct?
- How does a multimodal LLM process an image before the transformer works with it?
- Which tasks are typical for vision-language models?
- A tester gives a multimodal LLM a screenshot of the order page and the text of a defect report. Which task fits?
- What is a model optimized for during instruction tuning?
- In which order do the training stages of LLMs build on one another?
- What are instruction-tuned LLMs sometimes called to distinguish them from reasoning LLMs?
Generative AI in software testing
- How can an LLM help with analysing requirements?
- How do LLMs support the creation of test data?
- How does an LLM help when evaluating test results?
- Which tasks can an LLM take on in test automation?
- Which features characterize LLM-powered testing applications compared with chatbots?
- What is meant by prompt chaining when working with a chatbot?
- Regardless of access via chatbot or application, what most determines useful LLM results in testing?
Hard
Generative AI foundations
- A checking tool rejects commits as soon as a list of fixed conditions maintained by experts is violated, such as "no credentials in plain text". Which type of AI does this approach correspond to most closely?
- A tester sends the same prompt twice and receives two different sets of test cases. Which conclusions are correct?
- When analysing a very long log, an LLM no longer takes the early entries into account and contradicts them. What is the most likely cause?
- A test log is too long for the context window. Which measures help?
- A team wants to bring 500 test case titles into a fixed naming style. Which choice of model makes the most sense?
- Test cases for valid and invalid transition sequences are to be derived from a state model of a payment process with many conditions. Which kind of model fits best?
- A team uses a pure foundation LLM without further training to write test cases from a template. Instead of carrying out the instruction, the model often just continues the prompt. What explains this?
- A tester gives a multimodal LLM a wireframe of the registration page and the related user story. Which results can she reasonably expect?
- After comparing 40 screenshots with the acceptance criteria, a multimodal LLM reports three discrepancies. Which statements are correct?
- A search for "app closes during sign-in" also finds the older defect report "crash at login", although no word matches. What makes this possible?
- A team compares the token count of test reports of equal length in German and English. Which statements are correct?
- Which statements about using generative AI in testing compared with classical machine learning are correct?
- Which trade-offs apply when using a reasoning LLM instead of an instruction-tuned model?
- A company wants to run a model on its own test machines without the cloud, solely to bring error messages into a uniform format. Which kind of model is the obvious choice?
Generative AI in software testing
- Every night, the CI pipeline is to create draft test cases automatically from new user stories. Which approach fits?
- For which tasks is an AI chatbot better suited than a fully integrated LLM application?
- For a shipping cost rule, an LLM suggests "free from €40" as the expected result, while the requirement states €50. What is the right way for the tester to proceed?
- An acceptance criterion reads "The search returns results quickly". How can an LLM help in a useful way?
- An LLM is to suggest boundary values for the field "age" (valid from 18 to 65) using two-value boundary value analysis. Which values belong in a correct answer?
- After a nightly run, 300 automated tests have failed. How can an LLM support the team in a useful way?