Amazon AIF-C01 Practice Exam Questions & Answers

The verified answer is C. Faithfulness. The company is evaluating foundation models for a summarization task by using an LLM-as-a-judge evaluation job in Amazon Bedrock. AWS documentation states that with a model evaluation job that uses a judge model, Amazon Bedrock uses one LLM to score another model's responses and provide an explanation of how it scored each prompt and response pair. AWS also states that Amazon Bedrock provides built-in metrics that can be selected for these judge-based evaluation jobs.

Among the answer choices, Faithfulness is the only valid built-in metric for an Amazon Bedrock LLM-as-a-judge model evaluation job. AWS API documentation lists the valid built-in metric names for LLM-as-a-judge evaluation jobs, including Correctness, Completeness, Faithfulness, Helpfulness, Coherence, Relevance, FollowingInstructions, ProfessionalStyleAndTone, and responsible AI metrics such as Harmfulness, Stereotyping, and Refusal. The same documentation identifies Summarization as a valid task type for model evaluation jobs. Therefore, Faithfulness is a valid built-in metric for evaluating whether generated summaries remain grounded in the source input rather than introducing unsupported claims.

Option A. Context relevance and option B. Context coverage are incorrect for this question because AWS lists those as metrics for knowledge base retrieval-only evaluation jobs, not general LLM-as-a-judge model evaluation of foundation model summarization output. These metrics evaluate retrieved context, not the quality of a summarization model's generated response. Option D. RMSE is also incorrect because root mean square error is a regression metric. It is not a built-in LLM-as-a-judge metric for summarization in Amazon Bedrock.

Because the task is summarization and the evaluation job is LLM-as-a-judge, the correct built-in evaluation metric from the available choices is Faithfulness.

Comprehensive and Detailed Explanation From Exact AWS AI documents:

When a foundation model is customized directly in Amazon Bedrock, the correct way to use the customized model for inference is to purchase Provisioned Throughput.

Provisioned Throughput:

Enables consistent performance and predictable latency

Allows production use of customized Bedrock models

Is the required deployment mechanism for custom FMs in Bedrock

Why the other options are incorrect:

SageMaker endpoints (B) are not used for Bedrock-native custom models.

Model Registry (C) applies to SageMaker models.

Approval status (D) does not deploy or enable usage.

AWS AI document references:

Amazon Bedrock Custom Model Deployment

Provisioned Throughput for Foundation Models

Using Customized Models in Amazon Bedrock

Generative adversarial networks (GANs) are a type of deep learning model used for generating synthetic data based on existing datasets. GANs consist of two neural networks (a generator and a discriminator) that work together to create realistic data.

Option A (Correct): 'Generative adversarial network (GAN)': This is the correct answer because GANs are specifically designed for generating synthetic data that closely resembles the real data they are trained on.

Option B: 'XGBoost' is a gradient boosting algorithm for classification and regression tasks, not for generating synthetic data.

Option C: 'Residual neural network' is primarily used for improving the performance of deep networks, not for generating synthetic data.

Option D: 'WaveNet' is a model architecture designed for generating raw audio waveforms, not synthetic data in general.

AWS AI Practitioner Reference:

GANs on AWS for Synthetic Data Generation: AWS supports the use of GANs for creating synthetic datasets, which can be crucial for applications like training machine learning models in environments where real data is scarce or sensitive.

Ongoing pre-training when fine-tuning a foundation model (FM) improves model performance over time by continuously learning from new data.

Ongoing Pre-Training:

Involves continuously training a model with new data to adapt to changing patterns, enhance generalization, and improve performance on specific tasks.

Helps the model stay updated with the latest data trends and minimize drift over time.

Why Option B is Correct:

Performance Enhancement: Continuously updating the model with new data improves its accuracy and relevance.

Adaptability: Ensures the model adapts to new data distributions or domain-specific nuances.

Why Other Options are Incorrect:

A . Decrease model complexity: Ongoing pre-training typically enhances complexity by learning new patterns, not reducing it.

C . Decreases training time requirement: Ongoing pre-training may increase the time needed for training.

D . Optimizes inference time: Does not directly affect inference time; rather, it affects model performance.

Inference = applying a trained ML model to new, unseen data to make predictions, classifications, or generate outputs.

A is algorithm research, C refers to ensemble learning, D is data collection.

Reference:

AWS ML Glossary -- Inference