AI 原生开放教材

外观

AI Engineering Lab（24 周自学课程）系统课程·英文原文 ↗原件 ↓仓库 ↗

Week 20: AWS Bedrock & SageMaker AI ​

Part of AI Engineering Lab · Week 20 of 24 · Section: Cloud AI Platforms · Category: AWS 🎯 Use case: A Bedrock RAG support agent with Guardrails, plus the Great Platform Comparison matrix.

The problem ​

Three weeks, three clouds, one agent. On Foundry you learned that governance is a gateway; on Vertex you learned that a free tier is a liability and multimodal is a superpower. Now the agent lands on AWS, and the question changes from "how do I do X here" to the question a client or employer actually asks: which cloud should we standardize on? AWS's answer to "how do I keep the agent from saying the wrong thing" is different again, there is no gateway, no free sandbox, just IAM everywhere and a safety filter called Guardrails bolted onto every call.

AWS also forces you to name a distinction the other two clouds blurred: Bedrock is for consuming models; SageMaker is for owning them. Bedrock gives you one provider-neutral Converse API across Claude, Nova, Llama, and Mistral, with managed RAG (Knowledge Bases), agents, and guardrails layered on. SageMaker is the build/train/deploy-your-own layer you reach for only when fine-tuning leaves a measured gap. Knowing which layer you are standing on is the week's first skill, most AWS GenAI confusion is two overlapping products described as if they were one.

The deliverable this week is the Great Platform Comparison matrix: Foundry vs Vertex vs Bedrock, with rows for model access, unified API, agents, RAG, evals, fine-tuning, serving, governance, and pricing, every cell backed by a number you measured across Weeks 18 to 20. A comparison without per-cell evidence is a blog post; with it, it's a procurement decision. You close by naming, in one paragraph, which cloud you would standardize on and for what workload, with cost as an explicit input.

Objectives ​

  • [ ] By Friday you can call Bedrock models with the boto3 Converse API and run the Week 6 prompt suite across at least two models.
  • [ ] By Friday you can build a Bedrock Knowledge Base for RAG on shipping-policy docs and measure recall/groundedness with citations.
  • [ ] By Friday you can create a Bedrock Agent and a Guardrail, test blocked prompts, and report blocked/allowed counts.
  • [ ] By Friday you can publish the three-cloud comparison matrix (Foundry vs Vertex vs Bedrock) with cost, grounded in the numbers you measured across Weeks 18 to 20.

Day-by-day plan ​

DayStudyRun / buildShipTime
MonBedrock vs SageMaker; IAM least privilegeSet up the AWS account + least-privilege role; enable Bedrock modelsIAM policy + Model access2 to 3 h
TueConverse API; dated model IDsRun the Week 6 prompt suite across 2 models; price eachAccuracy/cost per model2 to 3 h
WedKnowledge Bases (managed RAG)Build a KB on the policy corpus; query with citationsRecall/groundedness2 to 3 h
ThuAgents + GuardrailsCreate an agent + guardrail; test blocked promptsBlocked/allowed counts2 to 3 h
FriThe matrixPublish the three-cloud comparison matrix with costMatrix + recommendation3 to 4 h
SatReviewTake the quiz (8/10)Quiz score in Notes1 h

Concepts ​

Read the shared mental model in reference/knowledge-base/12-cloud-platforms.md and the runbook in reference/platforms/aws-bedrock/README.md. AWS's generative-AI story is two complementary layers, and naming which one you stand on is the week's first skill:

Amazon BedrockSageMaker AI
JobConsume foundation models + managed GenAI featuresBuild, fine-tune, and serve your own models
AnalogyCall models as an APIOwn the model and the training
RAG/agents/guardrailsBuilt inVia JumpStart + endpoints
Reach for itAlways firstOnly when fine-tuning leaves a gap

The Converse API is the pedagogical star of the three clouds: one provider-neutral message/tool-calling format, so you swap the modelId and nothing else. That is why Week 20 runs the same Week 6 prompt suite across models and prices each, model selection becomes a measured loop, not a vibe.

The model catalog and dated IDs ​

Bedrock's catalog spans first-party and third-party models, all behind the same Converse surface:

FamilyVendorRole
Nova (Pro/Lite/Micro)AmazonCheapest, deep AWS integration; default for high-volume text
Claude (3.7 Sonnet / 4)AnthropicFrontier reasoning/coding; anchor of many agents
Llama (3.x/4)MetaOpen-weight, portable, self-host path
MistralMistralOpen-weight, efficient
TitanAmazonOlder first-party; embeddings still the RAG default

Two gotchas to internalize early. Bedrock model IDs carry dated suffixes (e.g. anthropic.claude-3-5-sonnet-20241022-v2:0) that get superseded often, copy the current ID from the console, never memorize it. And models are not enabled by default: an AccessDeniedException usually means you forgot Model access, not that your code is wrong.

Managed RAG via Knowledge Bases ​

Bedrock Knowledge Bases is fully managed RAG: point it at S3, and it chunks, embeds (Titan or Cohere), stores vectors (OpenSearch Serverless, Aurora, Pinecone, etc.), and exposes retrieve / retrieve_and_generate with citations. This is the same Week 7 job, grounding answers in the shipping-policy corpus, done by the platform. The two metrics you measure are recall (did retrieval surface the right document?) and groundedness (does the answer contain the ground-truth fact, i.e. come from a real citation rather than a hallucination?).

Agents, Guardrails, Prompt management, and AgentCore ​

Bedrock Agents = model + instructions + knowledge base + action groups (Lambda tools), with multi-agent collaboration. The Week 20 support agent wires track_shipment as an action group and the policy KB for grounding. Guardrails are configurable safety filters applied to model I/O:

Guardrail filterWhat it does
Denied topicsBlock defined off-topic subjects (e.g. competitor pricing)
Content filtersSeverity thresholds for hate/sexual/violence
PII redactionMask names, addresses, tracking identifiers
Custom word filtersBlock or flag specific terms

Prompt management versions your prompts as first-class, ARN-invokable resources, the platform-native answer to the Week 6 prompt-versioning habit. AgentCore is AWS's newer managed runtime for code-first agents (LangGraph, Strands), with memory, sessions, and a tool gateway, "run my own agent framework in production," worth knowing about but Bedrock Agents is the right entry point for this week. Around both sit Amazon Q (assistants you buy: Q Developer for code, Q Business for the workplace) and PartyRock (a free no-code playground, the best zero-friction "taste of Bedrock" before writing any boto3). The Week 20 support agent, drawn end to end:

mermaid
%%{init:{"theme":"base","fontFamily":"Helvetica Neue,Helvetica,Arial,sans-serif","flowchart":{"curve":"basis","padding":14,"nodeSpacing":45,"rankSpacing":55},"themeVariables":{"fontSize":"15px","background":"#FFFFFF","primaryColor":"#EEF2F7","primaryTextColor":"#14213D","primaryBorderColor":"#14213D","secondaryColor":"#FFF1E3","secondaryTextColor":"#14213D","tertiaryColor":"#E7F4F1","tertiaryTextColor":"#14213D","lineColor":"#64748B","textColor":"#14213D","edgeLabelBackground":"#FFFFFF","clusterBkg":"#F5F8FC","clusterBorder":"#94A3B8","nodeBorder":"#14213D","mainBkg":"#EEF2F7","titleColor":"#14213D"}}}%%
flowchart LR
    C["Client<br/>(boto3)"] --> GR["Guardrail<br/>denied topics · content filters · PII"]
    GR --> AG["Bedrock Agent<br/>model + instructions"]
    AG -->|"Converse API"| M["Foundation model<br/>Claude / Nova / Llama"]
    AG --> KB["Knowledge Base<br/>S3 → chunk → embed → vector store"]
    KB --> RT["retrieve_and_generate<br/>with citations"]
    RT --> M
    AG --> TG["Action group<br/>track_shipment (Lambda)"]
    GR --> OUT["Blocked / allowed<br/>counts"]

Guardrails sit at the edge, every request crosses them before the agent or the model spends a token, which is why the blocked/allowed counts are the Week 20 safety metric.

IAM least privilege, and a worked policy sketch ​

AWS has no API keys for models, everything is IAM, and that is itself the lesson: no keys in code, an explicit allow-list, aws configure (or aws sso login) for credentials. A worked example, a least-privilege policy scoped to what the labs actually need:

json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:ListFoundationModels",
        "bedrock:Converse",
        "bedrock:InvokeModel",
        "bedrock:Retrieve",
        "bedrock:RetrieveAndGenerate",
        "bedrock:CreateGuardrail",
        "bedrock:ApplyGuardrail",
        "bedrock:DeleteGuardrail"
      ],
      "Resource": "*"
    }
  ]
}

In production you would scope Resource to specific model ARNs and regions; for the labs the habits (explicit allow-list, least privilege, no * on admin actions) matter more than exact ARNs.

Pricing: on-demand vs provisioned vs cross-region ​

Bedrock is serverless by default. On-demand bills per token (input/output) or per image; Nova is typically cheapest, Claude frontier models most expensive. Provisioned Throughput reserves "model units" per hour at a commitment discount, it bills whether you use it or not, the AWS twin of Azure PTU. Cross-region inference routes across a pre-defined region set at the same token rates for higher throughput and resilience. A worked example of the commitment trap: if on-demand Claude costs ~$3 per 1M input tokens and you only push ~1M tokens/month, a provisioned model unit that reserves an hourly floor for ~$40/hour is a ~10,000× over-buy, provisioned throughput pays off only at steady, high, predictable volume, which ZoroLogistics does not have during a lab week.

How it breaks ​

Bedrock breaks in ways that look like your code but usually aren't. AccessDeniedException: either the model isn't enabled in Model access or the IAM policy lacks the action, check both before touching code. ThrottlingException: you exceeded the on-demand rate limit; add retry/backoff or use cross-region inference. ValidationException on modelId: the dated ID is stale, copy the current one from the console. Provisioned Throughput for the labs: it meters while idle, a weekend bill. No invocation logging: without logging to S3/CloudWatch you cannot audit spend. Agents/endpoints left running: they meter while idle, always run the cleanup cells. And the matrix's own failure mode: a comparison cell with no measured number, a matrix without per-cell evidence is a blog post, not a decision.

Notebook walkthrough ​

Two notebooks; both probe AWS (boto3.client("bedrock").list_foundation_models()) and fall back to a deterministic dry-run when credentials or model access are absent.

notebooks/01-bedrock-converse-and-knowledge-bases.ipynb sets REGION, probes AWS, then defines MODELS (defaulting to amazon.nova-lite-v1:0 and anthropic.claude-3-5-sonnet-20241022-v2:0) and a converse() helper that reads resp["output"]["message"]["content"][0]["text"]. It loads four BoL samples and reuses the Week 6 EXTRACT_PROMPT and the ten-field field_matches comparator to compute accuracy_for(model_id, docs), producing a per-model accuracy table. Then it defines kb_retrieve_and_generate (via bedrock-agent-runtime) and a local_retrieve fallback over data.policy_docs(), scores four RAG questions (R1, R4) for citation and groundedness, and prints the final numbers RECALL and GROUNDEDNESS. Correct dry-run output is RECALL: 1.000 GROUNDEDNESS: 1.000 (the local retriever maps all four keywords), which is the target a real KB must meet or beat it on the same question set.

notebooks/02-bedrock-agents-and-guardrails.ipynb creates a guardrail via bedrock.create_guardrail with a topicPolicyConfig (denying "OffTopic"), a contentPolicyConfig (INSULTS/HATE/VIOLENCE at HIGH strength), and a wordPolicyConfig (custom word competitorpricing + managed PROFANITY list). It then tests blocked prompts (insults, off-topic) and allowed prompts (tracking, refund) through apply_guardrail, with a local_guardrail fallback that blocks insult words and anything lacking a freight keyword. The agent cell calls bedrock-agent.create_agent when BEDROCK_AGENT_ROLE_ARN is set; the cleanup cells delete both the agent and the guardrail. The final cell prints BLOCKED: 3 ALLOWED: 2 in dry-run (three blocked, two allowed), which is the number you must reproduce against the live guardrail.

The use case (Friday) ​

Deliverable: the Great Platform Comparison matrix, Foundry vs Vertex vs Bedrock, with rows for model access, unified API, agents, RAG, evals, fine-tuning, serving, governance, and pricing, each cell backed by a number you measured across Weeks 18 to 20 (plus your Week 20 Bedrock RAG agent with Guardrails and its blocked/allowed counts).

Zorost gate: a stranger can read the matrix and your three deployment guides, and trace every "how do I do X here" claim to a runnable notebook with a printed number, same golden set, same seed, same metrics. You can state, in one paragraph, which cloud you would standardize on and for what workload, with cost as an explicit input.

Stretch: apply the Guardrail to the Knowledge Base response path (not just the agent) and show the blocked/allowed counts move for an off-topic query set; or add a third Bedrock model to the prompt-suite table and state the delta against the other two. Fast learners can also wire a track_shipment Lambda action group into the Bedrock Agent and run one end-to-end tracking query.

Common pitfalls ​

PitfallSymptomFix
Model not enabledAccessDeniedExceptionEnable it under Bedrock → Model access
Stale dated model IDValidationExceptionCopy the current ID from the console
Provisioned Throughput for the labsMetering while idleOn-demand only; never PTU for a lab week
Over the on-demand rate limitThrottlingExceptionRetry/backoff or cross-region inference
No invocation loggingCannot audit spendEnable model invocation logging to S3/CloudWatch
Resources left runningWeekend billRun the cleanup cells (agent + guardrail)
No AWS Budgets alertSurprise invoiceSet a $5 alert before any run
Matrix cell with no numberAnecdote, not evidenceEvery cell traces to a printed metric

Glossary ​

  • Amazon Bedrock: the managed foundation-model layer; call models as an API.
  • SageMaker AI: the build/train/deploy-your-own-model layer (JumpStart, HyperPod, customization).
  • Converse API: the provider-neutral message/tool-calling surface; swap modelId, nothing else.
  • Knowledge Base: managed RAG: S3 → chunk → embed → vector store → retrieve_and_generate.
  • Guardrail: configurable safety filter (denied topics, content filters, PII redaction) on model I/O.
  • Bedrock Agent: model + instructions + knowledge base + action groups (Lambda tools).
  • AgentCore: the managed runtime for code-first agents (LangGraph/Strands) with memory and sessions.
  • Recall: the fraction of questions where retrieval surfaced the correct source document.
  • Groundedness: the fraction of answers containing the ground-truth fact (cited, not hallucinated).
  • On-demand vs Provisioned Throughput: per-token vs reserved model-units-per-hour billing.
  • IAM least privilege: an explicit allow-list of bedrock:* actions, no keys in code.
  • Three-cloud matrix: the Week 20 capstone: Foundry vs Vertex vs Bedrock, every cell measured.

Self-check (quiz) ​

Take quiz.md, 10 questions tied to these concepts and the notebook code. The passing bar is 8/10; record the score in your Notes.

Exercises ​

Four graded exercises (easy / standard / stretch / portfolio) in exercises.md, from recording recall/groundedness, to a third-model comparison, to guardrailing the KB path, to publishing the three-cloud matrix itself. Hints are in the same file.

Sources ​