--- title: "Our production SDK hit 99.95% on ARC-AGI-3" date: 2026-08-27 description: "Strands Agents scored 99.95% on ARC-AGI-3 using the same minimal harness that developers ship to production every day" authors: - albert-zhao - chen-wu tags: - Announcement - Context Management - Benchmarks - Open Source draft: false --- import arcAgiVideo from './resources/arc-agi-3/arc-agi-3-example.mp4' Models beating video games can feel like AGI. However, we wondered how crucial Anthropic's harness was when they reported Fable 5 [completed](https://www.anthropic.com/research/claude-fable-5-mythos-5#:~:text=Fable%205%20is%20the,minimal%2C%20vision%2Donly%20harness.) Pokémon FireRed with a "minimal, vision-only harness". Fable 5 beat Pokémon FireRed with an agent harness that took screenshots, but it [failed](https://arcprize.org/results/anthropic-claude-fable-5) to complete every game puzzle in ARC-AGI, an intelligence benchmark designed so humans solve 100% of games but current models can't.
*Example of ARC-AGI-3 puzzle being done by a human. Our agent harness beat all levels without screenshots.*
Anthropic isn’t permitted to use a harness for its ARC-AGI set, so our team felt curious using our harness with a weaker model for ARC-AGI-3. We tested our open source Strands Harness SDK with Opus 5 and [achieved](https://arcprize.org/scorecards/8a10b024-3560-448f-ac31-becc48affe5b) a 99.95% ARC-AGI-3 score with all levels completed! Achieving a 99.95% score with Opus 5 and our harness, [compared](https://arcprize.org/results/anthropic-claude-opus-5) to Opus 5's current 30.16% score, highlights how much intelligence gets left on the table without a harness. OpenAI's latest model Astra [needed](https://arcprize.org/blog/astra) a custom harness to hit 99.99% on ARC-AGI-3. NVIDIA's AVO harness with Opus 5 [scored](https://developer.nvidia.com/blog/nvidia-avo-reaches-100-on-arc-agi-3-demonstrating-a-frontier-level-general-purpose-architecture-for-long-horizon-autonomous-agents/?ncid=so-twit-543600) a 100% on ARC-AGI-3. But, a key difference with our harness is we made the code [public](https://github.com/strands-labs/benchmark-harnesses/tree/main/arc-agi-3-agent). Through the power of a well-constructed agent loop and basic tools (e.g. `bash`, `grep`, file operations), our harness enabled Opus 5 to solve all levels without needing a single screenshot. ## Our agent did its own context engineering Our harness first renders ARC-AGI-3's numeric grid as text. The runner stores the latest board in `current_board.txt` and appends it to a `logs.txt` file to keep a history of the game. The agent can start to play the puzzle. The loop kicks off when the action queue is empty, triggering the runner to invoke the agent. Opus 5 can repeatedly search logs, write and run shell or Python scripts, check the results, then adjust its strategy. ```python def analyze( self, log_path: Path, action_num: int, retry_nudge: str = "", **kwargs: Any, ) -> Optional[dict[str, Any]]: log_path = Path(log_path) workspace = log_path.parent ... path_key = str(log_path) is_first = self._call_count.get(path_key, 0) == 0 self._call_count[path_key] = self._call_count.get(path_key, 0) + 1 actions_path = workspace / _ACTIONS_FILE if actions_path.exists(): actions_path.unlink() # PRO-LONG clears it every call active_route = self._route_for(path_key) agent = Agent( model=self._build_model(*active_route), tools=make_prolong_tools(workspace), system_prompt=self._build_system_prompt(), callback_handler=None, ) prompt = self._build_prompt( log_path, action_num, is_first, retry_nudge=retry_nudge, **kwargs ) t0 = time.time() result = None for attempt in range(_MAX_THROTTLE_RETRIES): try: result = agent(prompt) break except Exception as exc: ... if result is None: return None elapsed = time.time() - t0 text = str(result) meta = self._extract_usage(result, active_route[0]) if not actions_path.exists(): ... try: agent( f"You have not written {_ACTIONS_FILE} yet. Write it now with " 'the shape {"actions": ["ACTION1", "ACTION6(30,40)"]} — a list of ' f"1-{self._action_cap} actions to execute in order. Write the file; " "do not reply with the JSON only." ) except Exception as exc: ... ... actions = self._parse_actions_json_text( actions_path.read_text(errors="replace"), cap=self._action_cap ) ... return { "actions": actions, "hint": self._extract_tag(text, "PLAN") or text[-1500:], "plan": self._extract_tag(text, "PLAN") or "", "meta": meta, "cost": 0.0, } ```
*Source: [GitHub](https://github.com/strands-labs/benchmark-harnesses/blob/main/arc-agi-3-agent/prolong_agent/agent/strands_agent.py#L348-L494)*
What if the agent made a bad move? The runner records it in `logs.txt`. This allows our agent to learn and adjust, effectively running its own context engineering. The loop continues until the agent writes its next action plan in `actions.json`. Afterwards, the runner validates and executes the plan, kicking off another loop and recording the results. This cycle continues until every level is complete. ## What's next This design demonstrated a huge improvement in token-efficiency. The underlying PRO-LONG research, used in our harness, [found](https://arxiv.org/html/2607.20064v2#S2.SS2) that keeping history in an external log and retrieving only relevant slices used 4.2–5.8× fewer billed tokens than specialized harnesses (WorldModeler and Schema) with similar ARC-AGI-3 scores. The results we got back from our agent also revealed an impressive feat of self-directed context engineering. The agent made 734 scripts across 25 games after many loops. Imagine if this pattern extended into physical AI: A Strands Agent enabling a robot to maintain a working model of its environment, predict consequences for an action, then create a goal-oriented plan. We'd love to hear your thoughts. Find us in our [Discord](https://discord.gg/strands)!