11. Clean Up and Review Results
Stored responses created by lifecycle, stateful continuation, and background examples are tracked in STORED_RESPONSE_IDS. This final cell attempts to delete stored responses when cleanup is enabled, then prints the run summary and example-response gallery. Inspect warnings first; they usually identify endpoint configuration, model availability, or feature-support differences.
python
from __future__ import annotations
cleanup_rows = []
tracked_ids = list(dict.fromkeys(STORED_RESPONSE_IDS))
if not tracked_ids:
cleanup_rows.append({"response_id": "none", "status": "no stored responses tracked", "detail": ""})
elif not CLEAN_UP_STORED_RESPONSES:
for stored_id in tracked_ids:
cleanup_rows.append({"response_id": stored_id, "status": "skipped", "detail": "BEDROCK_CLEANUP_STORED_RESPONSES is disabled"})
record_check("Stored response cleanup", "skipped", cleanup_rows)
else:
for stored_id in tracked_ids:
try:
delete_result = delete_response(stored_id)
cleanup_rows.append({"response_id": stored_id, "status": "deleted", "detail": compact_text(to_dict(delete_result), 240)})
except Exception as exc:
cleanup_rows.append({"response_id": stored_id, "status": "warn", "detail": describe_api_error(exc)})
cleanup_status = "pass" if all(row["status"] == "deleted" for row in cleanup_rows) else "warn"
record_check("Stored response cleanup", cleanup_status, cleanup_rows)
print_label("Stored response cleanup")
display_wrapped_table(pd.DataFrame(cleanup_rows), max_col_width_px=520)
summary_df = pd.DataFrame(RESULTS_SUMMARY)
print_label("Run summary")
display_wrapped_table(summary_df, max_col_width_px=620)
print_label("Example responses")
print_response_gallery()Stored response cleanup
| response_id | status | detail |
|---|---|---|
| resp_cvhvh7y5ghwrpa35snvk4bzgcgthgxp4tgwkllmf5mrhs7dikfia | warn | \{'exception_class': 'AuthenticationError', 'status_code': 401, 'retryable': False, 'request_id': 'req_gkni5zyr7lkjkz2vfiwvkev2qgxs76crcwz5whhjrdkma7up3yta', 'message': 'Error code: 401 - \{'error': \{'code': 'invalid_api_key', 'message': 'The security token included in the request is invalid.', 'param': None, 'type': 'permission_denied_error'}}'\} |
| resp_vjrtvnakcgxjhnq5b7cj7rtowtdh7chkkf3aqwbdkynhjqiklp3a | warn | \{'exception_class': 'AuthenticationError', 'status_code': 401, 'retryable': False, 'request_id': 'req_vvwhsmp2rkrzwbkajqdpdod2o4j5xo2vxelzxbcp2fjan2ybwc2a', 'message': 'Error code: 401 - \{'error': \{'code': 'invalid_api_key', 'message': 'The security token included in the request is invalid.', 'param': None, 'type': 'permission_denied_error'}}'\} |
| resp_lmmtsvgk3ntolh5ci5vxccmsa6uxcgrsq7v54jpz7oewmociyesa | warn | \{'exception_class': 'AuthenticationError', 'status_code': 401, 'retryable': False, 'request_id': 'req_btgbpxspnokm3wzfndybnfuv3kjudxt3r2ihlsmru7ziisn52goa', 'message': 'Error code: 401 - \{'error': \{'code': 'invalid_api_key', 'message': 'The security token included in the request is invalid.', 'param': None, 'type': 'permission_denied_error'}}'\} |
Run summary
| name | status | detail |
|---|---|---|
| Endpoint shape | pass | https://bedrock-mantle.us-west-2.api.aws/openai/v1/responses |
| Model selection | pass | Using configured model; model-list metadata is not required for requests. |
| Error handling | pass | \{"normalized_fields": ["exception_class", "status_code", "retryable", "request_id", "message"], "retryable_status_codes": [408, 409, 429, 500, 502, 503, 504], "notes": "call_with_retries(...) uses this taxonomy for transient retry handling."\} |
| Text generation | pass | resp_naythl6fvzhoctlsdogd4vpr673q5ibagqqpiujbast3sy6viroa |
| Text generation | pass | \{"id": "resp_nmvqefzghd5hi67uwy4wfvhwnnzild3lslqsxqor3cat63kmucoq", "model": "openai.gpt-5.4", "status": "completed", "output_item_types": ["reasoning", "message"], "input_tokens": 177, "output_tokens": 129, "total_tokens": 306, "cached_input_tokens": 0, "reasoning_output_tokens": 18, "service_tier": "default"\} |
| Reasoning effort | pass | \{"id": "resp_nmvqefzghd5hi67uwy4wfvhwnnzild3lslqsxqor3cat63kmucoq", "model": "openai.gpt-5.4", "status": "completed", "output_item_types": ["reasoning", "message"], "input_tokens": 177, "output_tokens": 129, "total_tokens": 306, "cached_input_tokens": 0, "reasoning_output_tokens": 18, "service_tier": "default"\} |
| Responses lifecycle | pass | resp_cvhvh7y5ghwrpa35snvk4bzgcgthgxp4tgwkllmf5mrhs7dikfia |
| Response schema | pass | \{"id": "resp_cvhvh7y5ghwrpa35snvk4bzgcgthgxp4tgwkllmf5mrhs7dikfia", "model": "openai.gpt-5.4", "status": "completed", "output_item_types": ["message"], "input_tokens": 198, "output_tokens": 109, "total_tokens": 307, "cached_input_tokens": 0, "reasoning_output_tokens": 0, "service_tier": "default"\} |
| Usage metadata | pass | \{"id": "resp_cvhvh7y5ghwrpa35snvk4bzgcgthgxp4tgwkllmf5mrhs7dikfia", "model": "openai.gpt-5.4", "status": "completed", "output_item_types": ["message"], "input_tokens": 198, "output_tokens": 109, "total_tokens": 307, "cached_input_tokens": 0, "reasoning_output_tokens": 0, "service_tier": "default"\} |
| Prompt caching | pass | \{"id": "resp_q4akwbeynfwfwnt5i4tdwkpcgsffdu4lqvng7lnwaob53opswvwq", "model": "openai.gpt-5.4", "status": "completed", "output_item_types": ["reasoning", "message"], "input_tokens": 183, "output_tokens": 91, "total_tokens": 274, "cached_input_tokens": 0, "reasoning_output_tokens": 34, "service_tier": "default"\} |
| Service tier | pass | \{"id": "resp_q4akwbeynfwfwnt5i4tdwkpcgsffdu4lqvng7lnwaob53opswvwq", "model": "openai.gpt-5.4", "status": "completed", "output_item_types": ["reasoning", "message"], "input_tokens": 183, "output_tokens": 91, "total_tokens": 274, "cached_input_tokens": 0, "reasoning_output_tokens": 34, "service_tier": "default"\} |
| Reasoning effort | pass | \{"id": "resp_q4akwbeynfwfwnt5i4tdwkpcgsffdu4lqvng7lnwaob53opswvwq", "model": "openai.gpt-5.4", "status": "completed", "output_item_types": ["reasoning", "message"], "input_tokens": 183, "output_tokens": 91, "total_tokens": 274, "cached_input_tokens": 0, "reasoning_output_tokens": 34, "service_tier": "default"\} |
| Structured Outputs | pass | \{"ticket_id": "TICKET-7429", "category": "delivery_delay", "priority": "urgent", "customer_sentiment": "frustrated and time-sensitive", "summary": "Customer Maya Chen reports that ORDER-8831 is a replacement shipment for a previously damaged standing desk. The replacement is now 2 days late, carrier tracking has not updated, and she needs the desk delivered before Monday. She is requesting a supervisor callback and wants to know refund options if the replacement cannot arrive in time.", "require... |
| JSON mode | pass | \{"customer_name": "Maya Chen", "order_id": "ORDER-8831", "issue_summary": "Customer is asking about a delayed replacement order. The carrier tracking scan is stale and has not updated.", "next_step": "Handoff to support to investigate the carrier delay, verify shipment status, and provide Maya Chen with an update or resolution.", "metrics_to_watch": ["tracking_scan_recency", "carrier_exception_status", "replacement_order_delivery_eta", "customer_follow_up_time"]\} |
| Verbosity | pass | \{"compact_chars": 182, "detailed_chars": 332\} |
| Function calling | pass | \{"tool_choice_used": "required", "arguments": \{"order_id": "ORDER-8831"}} |
| Parallel tool calls | warn | \{"returned_order_ids": ["ORDER-8831"], "missing_order_ids": ["ORDER-2044"]\} |
| Custom tools | pass | \{"output_item_types": ["custom_tool_call"], "normalized_note": "ORDER_ID: ORDER-8831\nCUSTOMER_ID: CUST-1042\nISSUE: REPLACEMENT DELAYED\nCUSTOMER_REQUEST: CUSTOMER WANTS SUPERVISOR\nPOLICY_OPTION: OFFER EXPEDITED REPLACEMENT OR 15% CONCESSION"\} |
| Direct file inputs | warn | \{"message": "The request completed, but the response was not valid JSON.", "text_sample": "\{\"ticket_id\":\"TICKET-7429\",\"customer\":\"Maya Chen\",\"order_id\":\"ORDER-8831\",\"product\":\"Standing desk replacement\",\"issue\":\"Replacement for a damaged item is delayed and carrier scan has not moved\",\"requested_resolution\":\"Supervisor callback and refund options\",\"policy_options\":\"expedited replacement or 15% concession with agent approval after 48-hour delay\"\}", "error": "unhashable... |
| Stateful continuation | pass | resp_vjrtvnakcgxjhnq5b7cj7rtowtdh7chkkf3aqwbdkynhjqiklp3a |
| Stateless continuation | pass | \{"id": "resp_mezt6yqizyswuvujvudnonr34b73ndyyu2qsfncgtjyppzie5vva", "model": "openai.gpt-5.4", "status": "completed", "output_item_types": ["message"], "input_tokens": 255, "output_tokens": 78, "total_tokens": 333, "cached_input_tokens": 0, "reasoning_output_tokens": 0, "service_tier": "default"\} |
| Encrypted reasoning | pass | \{"encrypted_content_returned": true, "reasoning_item_count": 1\} |
| Prompt caching | pass | \{"first": \{"id": "resp_3w6r6ipbqa5z2max35awv3i23i5sjmfa33zzw3vhpgqxcvchhkgq", "model": "openai.gpt-5.4", "status": "completed", "output_item_types": ["message"], "input_tokens": 3970, "output_tokens": 66, "total_tokens": 4036, "cached_input_tokens": 0, "reasoning_output_tokens": 0, "service_tier": "default"\}, "second": \{"id": "resp_zzjeqttoswdjdwwl56xpolvly23w4h2n5dtsdoddgxqwfbkn7npq", "model": "openai.gpt-5.4", "status": "completed", "output_item_types": ["message"], "input_tokens": 3970, "outp... |
| Background mode | pass | \{"status_history": ["in_progress", "completed"], "id": "resp_lmmtsvgk3ntolh5ci5vxccmsa6uxcgrsq7v54jpz7oewmociyesa", "final_status": "completed"\} |
| Compaction | documented | \{"feature": "Compaction", "how_to_apply": "Summarize older support turns into durable facts, open questions, policy constraints, and next actions before continuing the workflow.", "brightcart_example": \{"durable_facts": ["Customer Maya Chen", "ORDER-8831", "replacement delayed", "carrier scan stale"], "policy_constraints": ["Do not promise refund without eligibility", "Offer expedited replacement or 15% concession after 48-hour delay with approval"], "next_action": "Check latest carrier scan and... |
| Latency runtime example | pass | \{"region_hint": "us-west-2", "base_url_host": "bedrock-mantle.us-west-2.api.aws", "sample_count": 3, "success_rate": 1.0, "completed_rate": 1.0, "avg_latency_seconds": 0.362, "p50_latency_seconds": 0.377, "p90_latency_seconds": 0.4, "total_output_tokens": 34, "total_tokens": 544\} |
| Throughput runtime example | pass | \{"region_hint": "us-west-2", "base_url_host": "bedrock-mantle.us-west-2.api.aws", "sample_count": 3, "success_rate": 1.0, "completed_rate": 1.0, "avg_latency_seconds": 0.362, "p50_latency_seconds": 0.377, "p90_latency_seconds": 0.4, "total_output_tokens": 34, "total_tokens": 544\} |
| Reliability runtime example | pass | \{"region_hint": "us-west-2", "base_url_host": "bedrock-mantle.us-west-2.api.aws", "sample_count": 3, "success_rate": 1.0, "completed_rate": 1.0, "avg_latency_seconds": 0.362, "p50_latency_seconds": 0.377, "p90_latency_seconds": 0.4, "total_output_tokens": 34, "total_tokens": 544\} |
| Region check | pass | \{"region_hint": "us-west-2", "base_url_host": "bedrock-mantle.us-west-2.api.aws", "sample_count": 3, "success_rate": 1.0, "completed_rate": 1.0, "avg_latency_seconds": 0.362, "p50_latency_seconds": 0.377, "p90_latency_seconds": 0.4, "total_output_tokens": 34, "total_tokens": 544\} |
| Stored response cleanup | warn | [\{"response_id": "resp_cvhvh7y5ghwrpa35snvk4bzgcgthgxp4tgwkllmf5mrhs7dikfia", "status": "warn", "detail": \{"exception_class": "AuthenticationError", "status_code": 401, "retryable": false, "request_id": "req_gkni5zyr7lkjkz2vfiwvkev2qgxs76crcwz5whhjrdkma7up3yta", "message": "Error code: 401 - \{'error': \{'code': 'invalid_api_key', 'message': 'The security token included in the request is invalid.', 'param': None, 'type': 'permission_denied_error'}}"}}, \{"response_id": "resp_vjrtvnakcgxjhnq5b7cj7rt... |
Example responses
| example | response_type | response |
|---|---|---|
| Endpoint verification | text | ok |
| First raw HTTPS request | text | Empathy: I’m sorry, Maya — your replacement order ORDER-8831 is delayed because the carrier reported a temporary transit hold at the regional sorting facility.\nAction: We’re monitoring the shipment closely and will send you an updated delivery estimate within 24 hours; if there’s no movement by then, we’ll review the next replacement or refund options with you. |
| SDK text generation | text | Use the Responses API to build a BrightCart support assistant that can answer customer questions, summarize policies, and guide users through common workflows like order tracking, refunds, and account updates. Ground the assistant in BrightCart documentation and connect it to relevant backend tools or APIs so it can retrieve live order data, check account status, and provide accurate, context-aware support responses. Design the experience around clear system instructions, structured tool calling, and conversation state management so the assistant stays on-brand, reliable, and safe when handling customer issues. |
| Create and retrieve response | text | goal: Help support agents explain delayed replacement orders, set expectations, and suggest next steps.\ndata needed: Order ID, replacement order status, shipment/tracking events, delay reason, estimated ship/delivery date, customer contact history, inventory/backorder status, and applicable refund or reship policy.\nhuman-review rule: Escalate to a human if the delay exceeds policy thresholds, tracking is inconsistent or missing, the order appears lost, the customer is high-risk or highly upset, or any refund/reship exception is requested. |
| Service tier and prompt cache request | text | Latency benefit: Prompt caching lets the BrightCart support assistant reuse previously processed context, reducing response time for repeated or similar requests.\nConsistency benefit: Prompt caching helps the BrightCart support assistant return more uniform answers by reusing the same established prompt context across interactions. |
| Structured ticket triage | json | \{\n "ticket_id": "TICKET-7429",\n "category": "delivery_delay",\n "priority": "urgent",\n "customer_sentiment": "frustrated and time-sensitive",\n "summary": "Customer Maya Chen reports that ORDER-8831 is a replacement shipment for a previously damaged standing desk. The replacement is now 2 days late, carrier tracking has not updated, and she needs the desk delivered before Monday. She is requesting a supervisor callback and wants to know refund options if the replacement cannot arrive in time.",\n "required_actions": [\n "Review ORDER-8831 shipment status and confirm last carrier scan/update.",\n "Contact carrier or open a trace/escalation for stalled tracking.",\n "Check expedited reshipment or alternative fulfillment options to meet the before-Monday deadline.",\n "Arrange supervisor callback per customer request.",\n "Review and communicate refund options, including refund\n... |
| JSON support handoff | json | \{\n "customer_name": "Maya Chen",\n "order_id": "ORDER-8831",\n "issue_summary": "Customer is asking about a delayed replacement order. The carrier tracking scan is stale and has not updated.",\n "next_step": "Handoff to support to investigate the carrier delay, verify shipment status, and provide Maya Chen with an update or resolution.",\n "metrics_to_watch": [\n "tracking_scan_recency",\n "carrier_exception_status",\n "replacement_order_delivery_eta",\n "customer_follow_up_time"\n ]\n\} |
| Compact policy guidance | text | BrightCart’s delayed-replacement policy lets customers keep using the original item until the replacement arrives, then return the defective product within the allowed return window. |
| Detailed policy guidance | text | 1. BrightCart sends replacements after customers return the original item and warehouse receipt is confirmed.\n2. This delay prevents duplicate shipments, verifies eligibility, and reduces fraud or inventory errors.\n3. Agents should explain timelines clearly, offer return instructions, and reassure customers once receipt is logged. |
| Order-status tool answer | text | Status: ORDER-8831 is delayed; carrier shows no movement for 36 hours at the Denver sort center, with promised delivery on 2026-06-01.\nNext best action: Monitor until the 48-hour threshold; if no movement then, contact the customer and offer either an expedited replacement or a 15% concession with agent approval. |
| Parallel order lookup fallback answer | text | Order statuses: ORDER-8831 is delayed and ORDER-2044 was delivered yesterday.\nShipping problems: Maya has one active shipping problem, because only ORDER-8831 is currently delayed while ORDER-2044 is already delivered. |
| Normalized support note | text | ORDER_ID: ORDER-8831\nCUSTOMER_ID: CUST-1042\nISSUE: REPLACEMENT DELAYED\nCUSTOMER_REQUEST: CUSTOMER WANTS SUPERVISOR\nPOLICY_OPTION: OFFER EXPEDITED REPLACEMENT OR 15% CONCESSION |
| Support transcript extraction text | text | \{"ticket_id":"TICKET-7429","customer":"Maya Chen","order_id":"ORDER-8831","product":"Standing desk replacement","issue":"Replacement for a damaged item is delayed and carrier scan has not moved","requested_resolution":"Supervisor callback and refund options","policy_options":"expedited replacement or 15% concession with agent approval after 48-hour delay"\} |
| Stateful support handoff | text | Ticket ID: TICKET-4812\nOrder ID: ORDER-8831\nCustomer Name: Maya Chen\nIssue: Replacement standing desk shipment for damaged delivery has had no carrier movement for 36 hours; customer is frustrated because this is the second attempt\nNext Best Action: Monitor until 48 hours without movement, then offer expedited replacement or 15% concession and escalate to Tier 2 Returns if needed |
| Stateless support handoff | text | Customer: Jordan Lee reported ORDER-7718 arrived with a cracked monitor stand.\nIssue: Damaged item; monitor stand is cracked on arrival.\nRequested Resolution: Customer wants a replacement shipped this week.\nOpen Question: Jordan asked whether the damaged item must be returned before replacement is sent.\nStatus: Damage claim captured and awaiting next-agent confirmation on replacement timing and return requirement. |
| State strategy recommendation | text | Recommendation: Stateless continuation\nReason: In a regulated support workflow, requiring names, order IDs, and refund context to be explicitly provided each turn improves controllability, auditability, and data-minimization, reducing the risk of unintended retention or cross-session leakage. |
| Prompt-cache token comparison | table | [\n \{\n "request":"first",\n "input_tokens":3970,\n "cached_input_tokens":0,\n "output_tokens":66,\n "total_tokens":4036\n \},\n \{\n "request":"second",\n "input_tokens":3970,\n "cached_input_tokens":0,\n "output_tokens":48,\n "total_tokens":4018\n \}\n] |
| Cached support-policy reply | text | Hi Maya, I’m sorry your replacement order ORDER-8831 is delayed. I’m checking the latest replacement and carrier status now so I can confirm the best next step for you without making you wait longer than necessary. |
| Background manager summary | text | theme: Shipping delays dominate, especially West Coast distribution lane.\nrisk: Rising dissatisfaction from delays, replacements, and return exceptions.\nnext action: Escalate West Coast lane issues and review holiday return policy. |
| Compacted support context | json | \{\n "feature": "Compaction",\n "how_to_apply": "Summarize older support turns into durable facts, open questions, policy constraints, and next actions before continuing the workflow.",\n "brightcart_example": \{\n "durable_facts": [\n "Customer Maya Chen",\n "ORDER-8831",\n "replacement delayed",\n "carrier scan stale"\n ],\n "policy_constraints": [\n "Do not promise refund without eligibility",\n "Offer expedited replacement or 15% concession after 48-hour delay with approval"\n ],\n "next_action": "Check latest carrier scan and supervisor callback status."\n \}\n\} |
| Endpoint responsiveness summary | json | \{\n "region_hint": "us-west-2",\n "base_url_host": "bedrock-mantle.us-west-2.api.aws",\n "sample_count": 3,\n "success_rate": 1.0,\n "completed_rate": 1.0,\n "avg_latency_seconds": 0.362,\n "p50_latency_seconds": 0.377,\n "p90_latency_seconds": 0.4,\n "total_output_tokens": 34,\n "total_tokens": 544,\n "samples": [\n \{\n "ok": true,\n "latency_seconds": 0.4,\n "output_tokens": 14,\n "total_tokens": 184,\n "status": "completed",\n "sample_output": "We apologize for the delay with your replacement order."\n \},\n \{\n "ok": true,\n "latency_seconds": 0.31,\n "output_tokens": 6,\n "total_tokens": 174,\n "status": "completed",\n "sample_output": "Resolution Rate"\n \},\n \{\n "ok": true,\n "latency_seconds": 0.377,\n "output_tokens": 14,\n "total_tokens": 186,\n "status": "completed",\n "sample_output": "I\u2019m e\n... |
| example | response_type | response | |
|---|---|---|---|
| 0 | Endpoint verification | text | ok |
| 1 | First raw HTTPS request | text | Empathy: I’m sorry, Maya — your replacement order ORDER-8831 is delayed because the carrier reported a temporary transit hold at the regional sorting facility.\nAction: We’re monitoring the shipment closely and will send you an updated delivery estimate within 24 hours; if there’s no movement by then, we’ll review the next replacement or refund options with you. |
| 2 | SDK text generation | text | Use the Responses API to build a BrightCart support assistant that can answer customer questions, summarize policies, and guide users through common workflows like order tracking, refunds, and account updates. Ground the assistant in BrightCart documentation and connect it to relevant backend tools or APIs so it can retrieve live order data, check account status, and provide accurate, context-aware support responses. Design the experience around clear system instructions, structured tool calling, and conversation state management so the assistant stays on-brand, reliable, and safe when handling customer issues. |
| 3 | Create and retrieve response | text | goal: Help support agents explain delayed replacement orders, set expectations, and suggest next steps.\ndata needed: Order ID, replacement order status, shipment/tracking events, delay reason, estimated ship/delivery date, customer contact history, inventory/backorder status, and applicable refund or reship policy.\nhuman-review rule: Escalate to a human if the delay exceeds policy thresholds, tracking is inconsistent or missing, the order appears lost, the customer is high-risk or highly upset, or any refund/reship exception is requested. |
| 4 | Service tier and prompt cache request | text | Latency benefit: Prompt caching lets the BrightCart support assistant reuse previously processed context, reducing response time for repeated or similar requests.\nConsistency benefit: Prompt caching helps the BrightCart support assistant return more uniform answers by reusing the same established prompt context across interactions. |
| 5 | Structured ticket triage | json | \{\n "ticket_id": "TICKET-7429",\n "category": "delivery_delay",\n "priority": "urgent",\n "customer_sentiment": "frustrated and time-sensitive",\n "summary": "Customer Maya Chen reports that ORDER-8831 is a replacement shipment for a previously damaged standing desk. The replacement is now 2 days late, carrier tracking has not updated, and she needs the desk delivered before Monday. She is requesting a supervisor callback and wants to know refund options if the replacement cannot arrive in time.",\n "required_actions": [\n "Review ORDER-8831 shipment status and confirm last carrier scan/update.",\n "Contact carrier or open a trace/escalation for stalled tracking.",\n "Check expedited reshipment or alternative fulfillment options to meet the before-Monday deadline.",\n "Arrange supervisor callback per customer request.",\n "Review and communicate refund options, including refund\n... |
| 6 | JSON support handoff | json | \{\n "customer_name": "Maya Chen",\n "order_id": "ORDER-8831",\n "issue_summary": "Customer is asking about a delayed replacement order. The carrier tracking scan is stale and has not updated.",\n "next_step": "Handoff to support to investigate the carrier delay, verify shipment status, and provide Maya Chen with an update or resolution.",\n "metrics_to_watch": [\n "tracking_scan_recency",\n "carrier_exception_status",\n "replacement_order_delivery_eta",\n "customer_follow_up_time"\n ]\n\} |
| 7 | Compact policy guidance | text | BrightCart’s delayed-replacement policy lets customers keep using the original item until the replacement arrives, then return the defective product within the allowed return window. |
| 8 | Detailed policy guidance | text | 1. BrightCart sends replacements after customers return the original item and warehouse receipt is confirmed.\n2. This delay prevents duplicate shipments, verifies eligibility, and reduces fraud or inventory errors.\n3. Agents should explain timelines clearly, offer return instructions, and reassure customers once receipt is logged. |
| 9 | Order-status tool answer | text | Status: ORDER-8831 is delayed; carrier shows no movement for 36 hours at the Denver sort center, with promised delivery on 2026-06-01.\nNext best action: Monitor until the 48-hour threshold; if no movement then, contact the customer and offer either an expedited replacement or a 15% concession with agent approval. |
| 10 | Parallel order lookup fallback answer | text | Order statuses: ORDER-8831 is delayed and ORDER-2044 was delivered yesterday.\nShipping problems: Maya has one active shipping problem, because only ORDER-8831 is currently delayed while ORDER-2044 is already delivered. |
| 11 | Normalized support note | text | ORDER_ID: ORDER-8831\nCUSTOMER_ID: CUST-1042\nISSUE: REPLACEMENT DELAYED\nCUSTOMER_REQUEST: CUSTOMER WANTS SUPERVISOR\nPOLICY_OPTION: OFFER EXPEDITED REPLACEMENT OR 15% CONCESSION |
| 12 | Support transcript extraction text | text | \{"ticket_id":"TICKET-7429","customer":"Maya Chen","order_id":"ORDER-8831","product":"Standing desk replacement","issue":"Replacement for a damaged item is delayed and carrier scan has not moved","requested_resolution":"Supervisor callback and refund options","policy_options":"expedited replacement or 15% concession with agent approval after 48-hour delay"\} |
| 13 | Stateful support handoff | text | Ticket ID: TICKET-4812\nOrder ID: ORDER-8831\nCustomer Name: Maya Chen\nIssue: Replacement standing desk shipment for damaged delivery has had no carrier movement for 36 hours; customer is frustrated because this is the second attempt\nNext Best Action: Monitor until 48 hours without movement, then offer expedited replacement or 15% concession and escalate to Tier 2 Returns if needed |
| 14 | Stateless support handoff | text | Customer: Jordan Lee reported ORDER-7718 arrived with a cracked monitor stand.\nIssue: Damaged item; monitor stand is cracked on arrival.\nRequested Resolution: Customer wants a replacement shipped this week.\nOpen Question: Jordan asked whether the damaged item must be returned before replacement is sent.\nStatus: Damage claim captured and awaiting next-agent confirmation on replacement timing and return requirement. |
| 15 | State strategy recommendation | text | Recommendation: Stateless continuation\nReason: In a regulated support workflow, requiring names, order IDs, and refund context to be explicitly provided each turn improves controllability, auditability, and data-minimization, reducing the risk of unintended retention or cross-session leakage. |
| 16 | Prompt-cache token comparison | table | [\n \{\n "request":"first",\n "input_tokens":3970,\n "cached_input_tokens":0,\n "output_tokens":66,\n "total_tokens":4036\n \},\n \{\n "request":"second",\n "input_tokens":3970,\n "cached_input_tokens":0,\n "output_tokens":48,\n "total_tokens":4018\n \}\n] |
| 17 | Cached support-policy reply | text | Hi Maya, I’m sorry your replacement order ORDER-8831 is delayed. I’m checking the latest replacement and carrier status now so I can confirm the best next step for you without making you wait longer than necessary. |
| 18 | Background manager summary | text | theme: Shipping delays dominate, especially West Coast distribution lane.\nrisk: Rising dissatisfaction from delays, replacements, and return exceptions.\nnext action: Escalate West Coast lane issues and review holiday return policy. |
| 19 | Compacted support context | json | \{\n "feature": "Compaction",\n "how_to_apply": "Summarize older support turns into durable facts, open questions, policy constraints, and next actions before continuing the workflow.",\n "brightcart_example": \{\n "durable_facts": [\n "Customer Maya Chen",\n "ORDER-8831",\n "replacement delayed",\n "carrier scan stale"\n ],\n "policy_constraints": [\n "Do not promise refund without eligibility",\n "Offer expedited replacement or 15% concession after 48-hour delay with approval"\n ],\n "next_action": "Check latest carrier scan and supervisor callback status."\n \}\n\} |
| 20 | Endpoint responsiveness summary | json | \{\n "region_hint": "us-west-2",\n "base_url_host": "bedrock-mantle.us-west-2.api.aws",\n "sample_count": 3,\n "success_rate": 1.0,\n "completed_rate": 1.0,\n "avg_latency_seconds": 0.362,\n "p50_latency_seconds": 0.377,\n "p90_latency_seconds": 0.4,\n "total_output_tokens": 34,\n "total_tokens": 544,\n "samples": [\n \{\n "ok": true,\n "latency_seconds": 0.4,\n "output_tokens": 14,\n "total_tokens": 184,\n "status": "completed",\n "sample_output": "We apologize for the delay with your replacement order."\n \},\n \{\n "ok": true,\n "latency_seconds": 0.31,\n "output_tokens": 6,\n "total_tokens": 174,\n "status": "completed",\n "sample_output": "Resolution Rate"\n \},\n \{\n "ok": true,\n "latency_seconds": 0.377,\n "output_tokens": 14,\n "total_tokens": 186,\n "status": "completed",\n "sample_output": "I\u2019m e\n... |