iSQI CT-AI Exam Dumps

A/B testing is a statistical testing method that compares two different versions of a system to determine which one performs better. In this scenario, the old NLP algorithm was rated for accuracy, and after the update, the new algorithm was also rated by users. A statistical test was performed to compare the two versions, which is the fundamental approach of A/B testing.

A/B testing is commonly used in:

User experience testing (e.g., comparing different versions of a website).

ML model evaluation (e.g., comparing two AI-based classifiers).

Performance assessment (e.g., determining if a new recommendation algorithm is more effective).

This approach allows for data-driven decisions, ensuring that any changes to the system result in meaningful improvements.

Reference from ISTQB Certified Tester AI Testing Study Guide:

Section 9.4 - A/B Testing states that A/B testing is used to compare updates in AI-based systems to determine if the newer version is better.

In AI-based systems, particularly those where the internal structure has been generated by another software system, the complexity often makes it difficult for human testers to analyze the inner workings. As per the ISTQB Certified Tester AI Testing (CT-AI) Syllabus:

Black-box testing is particularly useful when dealing with AI systems that have been generated by another system because:

It allows testing without requiring knowledge of the internal logic.

The AI model may be too complex for human testers to comprehend, making white-box testing ineffective.

Black-box testing evaluates the inputs and outputs, ensuring functional correctness without needing insight into how the system reaches a decision.

Why other options are incorrect?

A (Test automation and black-box testing): While automation is possible, black-box testing is not primarily about automation but about abstracting the internal complexity.

B (Understanding the logic of the software): This contradicts the premise of black-box testing, which is designed to test functionality without needing to understand the inner workings.

C (Checking transparency of the algorithm): Black-box testing does not check algorithm transparency---that would require white-box testing or explainability techniques.

Thus, the best choice is Option D, as black-box testing removes the need to analyze the internal structure of AI systems, making it the most appropriate testing method in this case.

Certified Tester AI Testing Study Guide Reference:

ISTQB CT-AI Syllabus v1.0, Section 8.5 (Challenges Testing Complex AI-Based Systems)

ISTQB CT-AI Syllabus v1.0, Section 8.6 (Testing the Transparency, Interpretability, and Explainability of AI-Based Systems)

According to the ISTQB Certified Tester AI Testing (CT-AI) syllabus, data acquisition, a critical step in data preparation for machine learning (ML) workflows, consists of three key activities:

Identification: This step involves determining the types of data required for training and prediction. For example, in a self-driving car application, data types such as radar, video, laser imaging, and LiDAR (Light Detection and Ranging) data may be identified as necessary sources.

Gathering: After identifying the required data types, the sources from which the data will be collected are determined, along with the appropriate collection methods. An example could be gathering financial data from the International Monetary Fund (IMF) and integrating it into an AI-based system.

Labeling: This process involves annotating or tagging the collected data to make it meaningful for supervised learning models. Labeling is an essential activity that helps machine learning algorithms differentiate between categories and make accurate predictions.

These activities ensure that the data is suitable for training and testing machine learning models, forming the foundation of data preparation.

Recognizing digits from a scan of handwritten numbers using machine learning is an example of classification. Here's a breakdown:

Classification: This type of machine learning involves categorizing input data into predefined classes. In this scenario, the input data (handwritten digits) are classified into one of the 10 digit classes (0-9).

Why Not Other Options:

Reinforcement Learning: This involves learning by interacting with an environment to achieve a goal, which does not fit the problem of recognizing digits.

Regression: This is used for predicting continuous values, not discrete categories like digit recognition.

Clustering: This involves grouping similar data points together without predefined classes, which is not the case here.

Overfitting occurs when a machine learning (ML) model learns patterns that are too specific to the training data, leading to a lack of generalization for new, unseen data. This means the model performs exceptionally well on the training data but poorly on validation or test data because it has memorized the noise and minor details rather than learning the underlying patterns.

Analysis of the Answer Options:

Option A: ''The model is not able to generalize to accommodate new types of data.''

This is the correct definition of overfitting. When a model cannot generalize beyond its training data, it struggles with new input, which results in overfitting.

Option B: ''The model is too simplistic for the data.''

This describes underfitting rather than overfitting. Underfitting happens when a model is too simple to capture the underlying patterns in the data.

Option C: ''The model is missing relationships between the inputs and outputs.''

This also aligns more with underfitting, where the model fails to capture important relationships in the data.

Option D: ''The model discards data it considers to be noise or outliers.''

While some ML models may ignore outliers, overfitting actually occurs when the model includes noise and outliers in its learning process rather than discarding them.

ISTQB CT-AI Syllabus Reference:

Overfitting Definition: 'Overfitting occurs when the model fits too closely to a set of data points and fails to properly generalize. It works well on training data but struggles with new data.'.

Testing for Overfitting: 'Overfitting may be detected by testing the model with a dataset that is completely independent of the training dataset'