# Quiz: Evaluating AI Agents

Let's assess your understanding of the agent tracing and evaluation concepts covered in this bonus unit.

This quiz is optional and ungraded.

### Q1: What does observability in AI agents primarily refer to?
Which statement accurately describes the purpose of observability for AI agents?

<Question
choices={[ 
  {
    text: "It involves tracking internal operations through logs, metrics, and spans to understand agent behavior.",
    explain: "Correct! Observability means using logs, metrics, and spans to shed light on the inner workings of the agent.",
    correct: true
  },
  {
    text: "It is solely focused on reducing the financial cost of running the agent.",
    explain: "Observability covers cost but is not limited to it."
  },
  {
    text: "It refers only to the external appearance and UI of the agent.",
    explain: "Observability is about the internal processes, not the UI."
  },
  {
    text: "It is concerned with coding style and code aesthetics only.",
    explain: "Code style is unrelated to observability in this context."
  }
]}
/>

### Q2: Which of the following is NOT a common metric monitored in agent observability?
Select the metric that does not typically fall under the observability umbrella.

<Question
choices={[ 
  {
    text: "Latency",
    explain: "Latency is commonly tracked to assess agent responsiveness."
  },
  {
    text: "Cost per Agent Run",
    explain: "Monitoring cost is a key aspect of observability."
  },
  {
    text: "User Feedback and Ratings",
    explain: "User feedback is crucial for evaluating agent performance."
  },
  {
    text: "Lines of Code of the Agent",
    explain: "The number of lines of code is not a typical observability metric.",
    correct: true
  }
]}
/>

### Q3: What best describes offline evaluation of an AI agent?
Determine the statement that correctly captures the essence of offline evaluation.

<Question
choices={[ 
  {
    text: "Evaluating the agent using real user interactions in a live environment.",
    explain: "This describes online evaluation rather than offline."
  },
  {
    text: "Assessing agent performance using curated datasets with known ground truth.",
    explain: "Correct! Offline evaluation uses test datasets to gauge performance against known answers.",
    correct: true
  },
  {
    text: "Monitoring the agent's internal logs in real-time.",
    explain: "This is more related to observability rather than evaluation."
  },
  {
    text: "Running the agent without any evaluation metrics.",
    explain: "This approach does not provide meaningful insights."
  }
]}
/>

### Q4: Which advantage does online evaluation of agents offer?
Pick the statement that best reflects the benefit of online evaluation.

<Question
choices={[ 
  {
    text: "It provides controlled testing scenarios using pre-defined datasets.",
    explain: "Controlled testing is a benefit of offline evaluation, not online."
  },
  {
    text: "It captures live user interactions and real-world performance data.",
    explain: "Correct! Online evaluation offers insights by monitoring the agent in a live setting.",
    correct: true
  },
  {
    text: "It eliminates the need for any offline testing and benchmarks.",
    explain: "Both offline and online evaluations are important and complementary."
  },
  {
    text: "It solely focuses on reducing the computational cost of the agent.",
    explain: "Cost monitoring is part of observability, not the primary advantage of online evaluation."
  }
]}
/>

### Q5: What role does OpenTelemetry play in AI agent observability and evaluation?
Which statement best describes the role of OpenTelemetry in monitoring AI agents?

<Question
choices={[ 
  {
    text: "It provides a standardized framework to instrument code, enabling the collection of traces, metrics, and logs for observability.",
    explain: "Correct! OpenTelemetry standardizes instrumentation for telemetry data, which is crucial for monitoring and diagnosing agent behavior.",
    correct: true
  },
  {
    text: "It acts as a replacement for manual debugging by automatically fixing code issues.",
    explain: "Incorrect. OpenTelemetry is used for gathering telemetry data, not for debugging code issues."
  },
  {
    text: "It primarily serves as a database for storing historical logs without real-time capabilities.",
    explain: "Incorrect. OpenTelemetry focuses on real-time telemetry data collection and exporting data to analysis tools."
  },
  {
    text: "It is used to optimize the computational performance of the AI agent by automatically tuning model parameters.",
    explain: "Incorrect. OpenTelemetry is centered on observability rather than performance tuning."
  }
]}
/>

Congratulations on completing this quiz! 🎉 If you missed any questions, consider reviewing the content of this bonus unit for a deeper understanding. If you did well, you're ready to explore more advanced topics in agent observability and evaluation!