Qwissit Artificial Intelligence

ISTQB CT-AI

Quality Characteristics and Acceptance Criteria

60 questions

Quality characteristics according to ISO/IEC 25059

  1. Which standard extends the quality model of ISO/IEC 25010 with AI-specific aspects? Easy
  2. From which two perspectives does ISO/IEC 25059 evaluate AI-based systems? Easy
  3. Why was the characteristic of functional correctness adjusted for AI-based systems? Easy
  4. What does the quality characteristic functional adaptability describe? Easy
  5. Which quality characteristic describes that a person or another system can step into the operation of an AI system in time? Easy
  6. In ISO/IEC 25010 of 2023, which term takes the place of what used to be called usability? Easy
  7. What does the quality characteristic transparency refer to in AI-based systems? Easy
  8. What does the quality characteristic AI robustness describe? Easy
  9. Which quality characteristic describes how well an operator can step into an AI system in time to avert damage or danger? Easy
  10. Which perspective does the quality characteristic societal and ethical risk mitigation belong to? Easy

AI and safety

  1. What characterizes safety-related systems? Easy
  2. How are the requirements of many AI-based systems implicitly provided? Easy
  3. Which tool is an example of explainable AI techniques? Easy
  4. How does the EU AI Act classify AI systems used as safety components, for example in aviation, medical devices or vehicles? Easy
  5. How do mature international functional safety standards deal with AI? Easy

Acceptance criteria

  1. What form do acceptance criteria for AI-based systems often take? Easy
  2. Which quality characteristic does the acceptance criterion "accuracy of 95% for an image recognition system" belong to? Easy
  3. An acceptance criterion states: "If an autonomous drone loses its GPS signal and reports an emergency, a supervisor can take over control within 0.5 seconds." Which quality characteristic is being tested? Easy
  4. An acceptance criterion requires the dashboard and API to return the unique version identifier of the deployed prediction model and a link to its documentation. Which quality characteristic is concerned? Easy
  5. An acceptance criterion states: "A chatbot must pass an internal red teaming assessment with at least 95% and refuse content that promotes violence, self-harm or hate speech." Which quality characteristic is being tested? Easy

Quality characteristics according to ISO/IEC 25059

  1. Which quality characteristics are new or modified in ISO/IEC 25059 compared with ISO/IEC 25010? Medium
  2. Which parent characteristic is functional adaptability assigned to as a new sub-characteristic? Medium
  3. Which parent characteristic is AI robustness assigned to as a new sub-characteristic? Medium
  4. Which parent characteristic is intervenability assigned to as a new sub-characteristic? Medium
  5. Which parent characteristics is transparency assigned to as a new sub-characteristic? Medium
  6. Under which circumstances should a robust AI system maintain its functional correctness? Medium
  7. Which areas does the characteristic societal and ethical risk mitigation cover, among others? Medium
  8. How do the quality characteristics of ISO/IEC 25059 affect testing? Medium

AI and safety

  1. Which additional challenges does AI bring to safety-related systems? Medium
  2. In connection with non-determinism, what can cause unexpected behavior even in rigorously tested AI models? Medium
  3. Why does self-learning undermine the safety case of a system? Medium
  4. Which is an example of a safety guard for self-learning systems? Medium
  5. Which limitations do explainable AI techniques such as LIME have? Medium

Acceptance criteria

  1. An acceptance criterion states: "A video streaming service adjusts its homepage to recommend at least 40% documentaries after a user has watched three full-length documentaries." Which characteristic is being tested? Medium
  2. An acceptance criterion states: "If the outdoor AI camera overheats, it switches to a simpler, power-saving model instead of failing." Which characteristic is being tested? Medium
  3. An acceptance criterion states: "Before the AI-controlled signal box carries out a switch setting marked as critical, the dispatcher has 20 seconds to stop it." Which characteristic is being tested? Medium
  4. Which acceptance criteria are examples for safety? Medium
  5. Which two standards provide the comprehensive framework for defining software quality in acceptance criteria for AI-based systems? Medium
  6. An acceptance criterion requires that the purchasing team of a bought-in ML model can trace where its training data comes from. Which characteristic is being tested? Medium
  7. Which acceptance criteria are examples for AI functional correctness? Medium

Quality characteristics according to ISO/IEC 25059

  1. A business unit requires for an ML classification system: "The system must not produce a single misclassification." How is this requirement to be assessed? Hard
  2. An image recognition system delivers good results on clean test images, but many errors on slightly noisy or deliberately manipulated images. Which quality characteristic is deficient? Hard
  3. A credit system rejects applications without applicants or clerks learning which model in which version is used and where the training data comes from. Which quality characteristic is affected? Hard
  4. How do user controllability and intervenability differ? Hard
  5. Which mappings of quality characteristic and perspective are correct according to ISO/IEC 25059? Hard
  6. A recommender system adapts to new user trends during operation, but loses accuracy against manipulated ratings. Which characteristics need to be weighed against each other here? Hard

AI and safety

  1. A manufacturer wants to demonstrate traceability from requirements to implementation for an AI-based brake assist system. Why is this difficult? Hard
  2. A rigorously tested AI model for an industrial control system reacts differently in the field to almost identical inputs. Which challenge is shown here? Hard
  3. A self-learning assistant in a control center was comprehensively tested before deployment. Which measures help so that no dangerous behavior creeps in during operation? Hard
  4. A team wants to use LIME for a safety-related AI system to make decisions traceable. What must be considered? Hard
  5. A medical device manufacturer wants to use an AI module as a safety component and rely solely on a mature functional safety standard. Which problems are there? Hard
  6. Why is developing safety-related systems with AI harder than without AI? Hard

Acceptance criteria

  1. An acceptance criterion states: "The farm control system notifies the farmer when a sensor's visual performance degrades by more than 30% and allows immediate manual override; without a response, it fully deactivates from 50%." Which characteristic does it belong to? Hard
  2. Which acceptance criteria test AI robustness? Hard
  3. Which acceptance criteria test intervenability? Hard
  4. An acceptance criterion for a nuclear power plant requires that 100% of the input-output relationships of the ML model can be mapped by an explainability tool with an average accuracy of at least 99.9%. Which area does it belong to? Hard
  5. An acceptance criterion for a sentencing system requires that it does not discriminate between population groups based on a specified fairness metric. Which statements about this are correct? Hard
  6. A project manager wants to carry out the acceptance of an AI system like classic software using individual test cases with "pass/fail". What speaks against this? Hard
  7. Which acceptance criterion would be suitable for the functional adaptability of an engine management system? Hard
  8. A defect prediction system is to identify at least 90% of the actually defective modules as defective. Which statements about this are correct? Hard

Certifications & exams

Which exam comes next?

You can already practise for 19 exams, and 21 are up for a vote. See all certifications with their official information, cast your vote or suggest your exam.

See all certifications