Inspect feature delivery provenance across sessions and commits

Traceability

• Spec ID: harness-inspector
• Status: Implemented (Draft PR design review requested)

Intent

Give maintainers one read-only Harness Inspector that can move between a spec-driven feature tree and a date-oriented delivery window without changing the underlying evidence. The Inspector should connect declared product intent to privacy-safe prompts, coding-agent sessions, complete retained tool-call traces, and correlated commits so a reviewer can inspect both the macro delivery story and the micro session activity that produced it.

The feature tree is a small Markdown checklist intended to be generated by a future Skill and reviewed in Git. It declares product structure and explicit references; it does not manufacture session-to-commit evidence. Date mode is an independent picker over observed timestamps and remains useful when a feature tree is absent or incomplete.

Acceptance Scenarios

• AC-1: A versioned parser accepts a Markdown checklist tree using two-space indentation and - [ ] / - [x] todo state. Internal items become Features, leaves become Stories, and optional \{#id\} anchors provide explicit stable ids; otherwise ids are derived from English titles. The legacy typed-heading form remains readable. Invalid indentation, malformed checklist items, and duplicate ids produce line-local diagnostics.
• AC-2: harness-inspector render reads a bounded workspace/session window, composes only public commit-session-link and session-analysis surfaces, and writes one self-contained HTML report. --since/--until, --stage, --platform, commit/session bounds, and an optional feature-tree path select the inspected phase.
• AC-3: The report preserves both a declared product hierarchy and observed date navigation. The checklist renders as a semantic, collapsible Delivery Tree with nested branches, visible complete/todo state, and only the selected branch expanded by default. Selecting any node scopes the workbench without conflating tree state with evidence confidence. Each selected Story/session renders a prompt, normalized activity, and commit/file workbench. The whole Delivery Tree can collapse to a reversible rail, and the Prompt/Activity and Activity/Delivery boundaries are resizable by pointer or keyboard without introducing page-level horizontal overflow. The workspace header is one compact toolbar: breadcrumb owns the selected-scope title, only non-zero scope metrics render, and evidence labels stay with the evidence they describe instead of occupying a separate global legend row.
• AC-4: The report distinguishes user demand, agent activity, commit delivery, and changed-file evidence in simultaneous lanes without presenting time, text similarity, or shared paths as proof that a session produced a commit.
• AC-5: Each retained session exposes its complete NormalizedToolActivityV1 as a horizontally scrollable Swimlane Bubble Chart. The chart groups provider-neutral action labels, retains every individual call step, and uses versioned normalized families only for stable color semantics. Following Entire's compact transcript boundary, every tool step retains a tool name and a bounded redacted detail when safe; shell transports such as exec_command, Bash, and shell are classified in the data layer as actions such as search, read, edit, test, Git inspection, delivery, or generic execution. The renderer never parses raw command payloads. Hovering, focusing, or selecting a bubble exposes the retained host tool, timing, bounded detail, and attributed files without a redundant call list or nested chart card. Expanding the chart enters a focus layout that collapses the Commits/files lane to a reversible rail and restores it when the activity detail closes.
• AC-6: Commit changed files may always be shown as commit evidence. A file may be labeled as a session/commit overlap only when a repository-relative path is observed on both sides. Any context action remains read-only and must not be called recovery or resume until its owning checkpoint capability exists.
• AC-7: Reader output keeps repository paths relative, sanitizes prompt text and labels, escapes every HTML/attribute boundary, and never contains raw tool inputs/outputs, transcript bodies, credentials, absolute home paths, or unbounded native identifiers.
• AC-8: Focused tests cover feature-tree parsing, selection, evidence-kind projection, HTML interaction hooks, complete tool-call retention, CLI help, and a temporary Git fixture. A real Better Harness session report is checked in desktop and narrow browser viewports after the data-contract gaps below have dedicated regression fixtures.
• AC-9: Open session renders the hydrated session as chronological turns, reusing the public Entire-style turn model: each user prompt is followed by its intermediate assistant notes and tool calls, then its final assistant response, with correlated commits placed in the nearest turn window. Visible dialogue is credential/path sanitized; hidden reasoning, system/developer messages, and raw tool inputs/outputs remain excluded. Missing or truncated content is labeled instead of reconstructed. Intermediate responses render as light timeline prose; each turn's tool-call group is collapsed by default and expands in place without removing normalized action, host-tool, detail, timing, or file evidence.

Feature Tree Contract

The canonical input is an ordinary Markdown checklist. Two-space indentation is the parent/child relationship, [x] means complete, and [ ] means todo. Internal items become Features and leaves become Stories. An optional \{#id\} anchor overrides the id derived from an English title. The previous typed Feature / Story heading form remains a backward-compatible input.

- [ ] Better Harness
  - [ ] Inspector {#harness-inspector}
    - [ ] Inspect one delivery stage
    - [ ] Review complete session provenance

The future generator Skill should emit only this reviewed declarative artifact. It may propose refs from visible evidence, but must mark missing links instead of inventing ids or treating semantic similarity as provenance. Skill creation, installation, and autonomous activation are separate from this runtime slice.

Real-world Evidence Audit (2026-08-12)

The first audit used the local Better Harness repository, 17 Codex sessions and 60 recent commits from 2026-08-01 through 2026-08-12. It found these blocking data-contract problems before a second UI iteration:

1. Codex tool identity required batch-hydration repair. The initial 17 sessions collapsed most calls to the outer exec/wait transport. The implemented session projection now requests command text only as transient attribution input, resolves nested tool identity, then discards the command. Real output exposes provider-neutral actions while retaining sanitized host tool identities such as exec_command, apply_patch, browser, and test capabilities without retaining raw tool payloads.
2. Session-side file evidence required structured extraction. The initial sample exposed zero usable session files. The implemented projection now retains every safe repository-relative path from direct file facts and multi-file apply_patch headers. Shared paths are displayed as observed same-path evidence or same-file history, never as proof of authorship.
3. Prompt records are observations, not user turns. The initial sample contained 140 raw user-prompt records, 118 privacy-safe records, and 53 distinct safe requests. The report now separates prompt observations, normalized user turns, and the bounded retained display summaries.
4. Historical Feature/Story identity is sparse. Of 85 specs, 84 have a Traceability section but only 16 declare a Story; among the 30 specs since 2026-08-01 only five do. Of 60 recent commits, 46 use Conventional Commit syntax, 22 touch a spec, and 14 are merges. Commit scopes and filenames can suggest a grouping, but cannot authoritatively reconstruct Feature/Story.
5. Recovery is not implemented. HarnessCheckpointV1 is currently a Draft specification with no runtime owner under scripts/. Existing Entire checkpoint resolution belongs to commit/session evidence and is not Better Harness resume authority. The Inspector may later prepare a bounded context packet, but must not expose a working “Recover” action today.

These observations imply one evidence model for the next slice:

• declared: reviewed Feature/Story/spec/session/commit refs;
• observed: typed host evidence or an exact repository-relative file overlap;
• candidate: conventional-commit/spec/time hints awaiting review; and
• unmapped: real sessions or commits with no defensible product owner.

Feature and Story should therefore remain distinct in the data contract while rendering as one compact tree. Feature is a navigation aggregate; Story is the smallest reviewed delivery context. A generated tree must preserve candidate and unmapped rows instead of silently promoting inference to declaration.

Non-goals

• Writing Git notes, trailers, branches, commits, checkpoints, or native host session state.
• Restoring a worktree, replaying tool calls, resuming a native Codex session, or presenting a task packet as a rollback/recovery authority.
• Treating HarnessCheckpointV1 as a transcript, Git rollback point, Session Viewer, feature identity, or mutation authority.
• Creating or installing the future feature-tree generator Skill in this slice.
• Replacing TaskEpisode, PatternCandidate, report-local Finding, or longitudinal HarnessIssue with UI nodes.
• Fetching issues, Stories, or specs from a remote tracker during rendering.
• Persisting raw prompts, reasoning, commands, tool payloads, or assistant transcript bodies in the feature tree.

Plan and Tasks

1. Add scripts/harness-inspector/feature-tree.mjs as the strict parser and selection owner for FeatureTreeV1.
2. Add a pure HarnessInspectorReportV1 builder that joins the feature tree, bounded privacy-safe session summaries, and CommitSessionLinkV1 evidence without widening those source contracts.
3. Render checklist hierarchy and todo state as an accessible collapsible tree, retaining node selection as the workbench scope interaction.
4. Map the complete normalized activity projection into the established Canvas SDK SwimlaneBubbleChart semantics without copying raw transcript lanes.
5. Render a self-contained Delivery Workbench with user turns on the left, normalized activity in the center, and commits/changed files on the right. Keep the source HTML shell, CSS, and browser interaction script as separate capability-owned UI assets; the Node renderer injects sanitized projections into those assets and still emits one portable, self-contained HTML file. Use a compact breadcrumb header and open each real session in a read-only, chronological turn timeline with prompt/response/intermediate-step/commit/tool filters. Unretained response bodies must appear as unavailable rather than reconstructed prose.
6. Add a help-first CLI and register harness-inspector as an advanced command.
7. Add one generic feature-tree template and a current-project candidate tree whose inferred nodes remain visibly candidate until maintainer review.
8. Render real local Codex evidence, verify both picker modes and the largest retained trace in a browser, then run the documentation graph, full tests, packaging, and Review Readiness checks.

Test and Review Evidence

• AC-1/AC-3/AC-4: node --test test/harness-inspector.test.mjs
• AC-2/AC-8: node --test test/better-harness-cli.test.mjs
• AC-5: focused Inspector and test/commit-session-link.test.mjs coverage with more than 1,000 calls retained when no explicit limit is requested.
• AC-6/AC-7: final-HTML assertions for mode controls, Swimlane chart templates, evidence labels, extracted UI asset assembly, escaping, credential redaction, and home-path absence.
• AC-8: node scripts/doc-link-graph/cli.mjs skills/better-harness, node --test test/doc-link-graph.test.mjs, npm test, and npm run pack:verify; then a browser pass against a generated real report.
• Correlation risk: heuristic commit matches remain labeled with their confidence and evidence. Only declared refs, typed trailers, or resolved checkpoint metadata may appear as explicit relationships.
• Scale risk: date windows can contain many sessions and calls. CLI bounds the session/commit population, picker filtering limits visible rows, each complete trace owns horizontal overflow inside the Activity lane, and session tool-call groups start collapsed.
• Privacy risk: the Inspector consumes sanitized summaries and the public ToolCallTraceV2 projection. Command text is transient normalization input for provider-neutral action/detail extraction and is discarded before report projection; adversarial fixtures check the final HTML.
• Visual risk: the three-lane layout is intentionally delivered in a Draft PR for design review. The tool-call Swimlane is visually verified with real Codex data; broader density, responsive lane behavior, and the continuation dialog remain follow-up UX work rather than provenance claims.