Qwissit Artificial Intelligence

ISTQB CT-AI

Input Data Testing

60 questions

Input data risks

  1. What is the objective of input data testing? Easy
  2. Which mitigation fits the risk "training data acquired from untrustworthy sources"? Easy
  3. Which mitigation fits sub-optimal feature selection? Easy

Testing for bias

  1. What is meant by bias in an ML system? Easy
  2. What are the two main sources of bias in ML systems? Easy
  3. Which is an example of algorithmic bias? Easy
  4. What are counterfactuals in disparate impact analysis? Easy

Data pipeline and representativeness

  1. How does the layered approach to testing a data pipeline begin? Easy
  2. How does system testing of a fully assembled data pipeline begin? Easy
  3. What does testing of training pipelines often focus on, unlike operational pipelines? Easy
  4. What does data representativeness testing determine? Easy
  5. When should data representativeness testing be performed? Easy
  6. Which formal statistical tests are suitable for comparing distributions in representativeness testing? Easy

Dataset and label checks

  1. What does dataset constraint testing check? Easy
  2. Which constraint checks whether a value lies within a given range? Easy
  3. Why is dataset constraint testing usually automated? Easy
  4. In which type of machine learning is label correctness particularly essential? Easy
  5. What happens in multiple annotation? Easy
  6. Which metrics can be used to measure inter-annotator agreement (IAA)? Easy
  7. Which constraint checks whether an attribute value matches the specified data type, for example an integer instead of a string? Easy

Input data risks

  1. Which mitigations help against poisoned training data? Medium
  2. Against which risks does dataset constraint testing help? Medium
  3. Against which risks does data representativeness testing help? Medium

Testing for bias

  1. Which test approaches are suitable for detecting bias? Medium
  2. How does dynamic testing help detect bias in an ML model? Medium
  3. In which order do the steps of disparate impact analysis take place? Medium
  4. What must be considered in disparate impact analysis with combinations of sensitive attributes and with unrealistic examples? Medium
  5. What does a review of a dataset's documentation examine when testing for bias? Medium

Data pipeline and representativeness

  1. What is checked in component testing of a data pipeline? Medium
  2. What does fault injection testing of a data pipeline measure? Medium
  3. What is back-to-back testing used for in system testing of a data pipeline? Medium
  4. What are configuration management reviews used for when testing data pipelines? Medium
  5. Which three main steps does data representativeness testing comprise? Medium
  6. How can the expected operational data distributions and critical edge cases be identified when defining the target population? Medium

Dataset and label checks

  1. Which constraints are multi-value constraints? Medium
  2. What does the useful constraint check? Medium
  3. Which constraints are comparison constraints? Medium
  4. What can low inter-annotator agreement (IAA) scores indicate? Medium
  5. What can a data point with high loss during model training indicate? Medium
  6. What can a low prediction confidence of a trained model for a data point mean? Medium

Input data risks

  1. A team finds that after an uncontrolled update of a transformation script, the data pipeline delivers wrong values and becomes slower in operation. Which mitigation fits these risks? Hard
  2. Which mappings of input data risk and mitigation are correct? Hard

Testing for bias

  1. A credit model is to be checked for bias. The team generates applications that are identical except for gender and finds over many tests that the decisions differ significantly. Which statements are correct? Hard
  2. In a disparate impact analysis for mortgages, a tester changes the age in counterfactual applications to five years. What is the problem? Hard
  3. An applicant filter disadvantages female applicants because the training data comes from a time when hardly any women were hired. Which statements about this are correct? Hard
  4. A team wants to find out whether its model produces biased results, but does not know whether the cause lies in the data or in model development. Which approach detects bias regardless of its cause? Hard

Data pipeline and representativeness

  1. Which mappings of pipeline test and test level are correct? Hard
  2. A new version of the operational pipeline is to be introduced without degrading the live data stream. Which tests in production are suitable for this? Hard
  3. A team tests the training pipeline of an exploratory prototype and the operational pipeline of the product with an identical strategy. What would be better? Hard
  4. A skin cancer classifier was trained only with images from a single clinic and is to be used nationwide. Which steps of representativeness testing are appropriate here? Hard
  5. A model was trained and deployed after a successful representativeness test. What remains to be done in operation? Hard

Dataset and label checks

  1. In a dataset of election results, the vote shares of all parties in one constituency add up to 104%, although they should be exactly 100%. Which constraint detects this defect? Hard
  2. A customer dataset contains the entries "age = −3" and "age = thirty", and for one customer the age is missing entirely. Which constraints detect these defects? Hard
  3. A training dataset contains the column "customer number" with only unique values as well as several nearly identical rows. Which constraints are triggered here? Hard
  4. In a dataset about software modules, one module has more defective lines of code than lines of code in total. Which constraint detects this? Hard
  5. A rule states: all students with a mark at least 1.33 standard deviations above the mean have the grade "A". Which type of constraint is this, and which group does it belong to? Hard
  6. A team only has budget to have a small part of 100,000 images reviewed by experts. Which images should preferably be reviewed? Hard
  7. Bounding boxes in an object detection dataset are to be checked automatically. Which approach and which rule fit? Hard
  8. A team wants to ensure label quality throughout the project. Which combination is often most effective? Hard
  9. A team has a new labeled dataset of traffic scenes and knows a comparable public dataset. For "pedestrian", the label share is five times as high as in the comparison dataset. Which approach is this, and what follows from it? Hard

Certifications & exams

Which exam comes next?

You can already practise for 19 exams, and 21 are up for a vote. See all certifications with their official information, cast your vote or suggest your exam.

See all certifications