4.5 Concurrency Control

Concurrent tool execution follows strict rules:

• Concurrency-safe tools can run in parallel: Tools with isConcurrencySafe(input) === true (such as FileReadTool, GrepTool, GlobTool) can execute simultaneously
• Non-concurrency-safe tools run serially: Tools with isConcurrencySafe(input) === false (such as FileEditTool, BashTool write commands) must execute exclusively
• isReadOnly is only an intuition: read-only usually implies concurrency-safe, but they are two separate methods on the interface—what actually gates concurrent scheduling is isConcurrencySafe, not isReadOnly

Tool orchestration is managed by the runTools() function in src/services/tools/toolOrchestration.ts. It does not filter out all read-only tools at once and run them in a single Promise.all; instead it groups consecutive concurrency-safe calls into a batch that runs in parallel, in output order, and starts a new serial batch whenever it hits a non-concurrency-safe call:

// The heart of partitionToolCalls: batch by isConcurrencySafe, preserving order
function partitionToolCalls(toolUses) {
  return toolUses.reduce((batches, toolUse) => {
    const tool = findToolByName(tools, toolUse.name)
    const parsed = tool?.inputSchema.safeParse(toolUse.input)
    const safe = parsed?.success
      ? Boolean(tool.isConcurrencySafe(parsed.data))  // the test is concurrency-safety, not read-only
      : false
    // Consecutive concurrency-safe calls merge into the same batch; otherwise start a new one
    if (safe && batches.at(-1)?.isConcurrencySafe) {
      batches.at(-1).blocks.push(toolUse)
    } else {
      batches.push({ isConcurrencySafe: safe, blocks: [toolUse] })
    }
    return batches
  }, [])
}

// Each concurrency-safe batch runs in parallel (runToolsConcurrently),
// throttled by getMaxToolUseConcurrency() (default 10); non-safe batches run serially.

StreamingToolExecutor: Streaming Parallel Execution

The static orchestration strategy above has a limitation: it must wait for the model to fully output all tool_use blocks before starting execution. In reality, the model's streaming output takes 5-30 seconds, and a tool_use block may be complete within the first few seconds of streaming—why wait until the end?

StreamingToolExecutor (src/services/tools/StreamingToolExecutor.ts, approximately 530 lines) is designed precisely for this. While the model is streaming output, as soon as it detects a complete tool_use block, it immediately starts execution:

// Tools go through 4 states in StreamingToolExecutor
type ToolStatus = 'queued' | 'executing' | 'completed' | 'yielded'

// Tracking information for each tool
type TrackedTool = {
  id: string
  block: ToolUseBlock
  assistantMessage: AssistantMessage
  isConcurrencySafe: boolean
  results?: Message[]
  pendingProgress: Message[]    // Progress messages emitted immediately, not waiting for final result
}

Concurrency control rules are embedded directly within the executor:

// Can a tool be executed?
private canExecuteTool(isConcurrencySafe: boolean): boolean {
  const executingTools = this.tools.filter(t => t.status === 'executing')
  return (
    executingTools.length === 0 ||
    (isConcurrencySafe && executingTools.every(t => t.isConcurrencySafe))
  )
}

The rule is simple: if no tools are currently executing, any tool can start; if tools are executing, a new tool can only start when both itself and all currently executing tools are marked as concurrency-safe. Non-concurrency-safe tools must execute exclusively.

A timeline comparison demonstrates the effect of this optimization:

Serial execution (waiting for all tool_use to complete):
[==========API streaming output==========][tool1][tool2][tool3]

Streaming parallel execution (StreamingToolExecutor):
[==========API streaming output==========]
   [tool1]              ← Starts immediately after tool_use_1 completes
      [tool2]           ← Leverages streaming window (5-30s) to cover tool latency
         [tool3]
[======Results ready by the time API output completes======]

In typical scenarios, tool execution latency is about 1 second, while model streaming output lasts 5-30 seconds. This means most tool execution can be completely hidden within the streaming window, and the total latency perceived by the user approaches pure API call time.

Another key design is the progressAvailableResolve wake-up signal. The result consumer (getRemainingResults()) works in an event-driven manner—when new results or progress are ready, the executor wakes the consumer by resolving a Promise, avoiding polling overhead. pendingProgress messages (such as BashTool's stdout stream) are emitted to the UI immediately, without waiting for the tool to finish.

Partitioning Algorithm and Concurrency Limits

Inside StreamingToolExecutor, tools are divided into two execution categories:

1. Concurrency-safe tools (isConcurrencySafe: true): Can execute simultaneously with other concurrency-safe tools. Typical examples are FileReadTool, GrepTool, GlobTool—they only read data and don't interfere with each other.
2. Non-concurrency-safe tools: Must execute exclusively; no other tools can execute simultaneously while they run. FileEditTool and BashTool write operations fall into this category.

The executor's scheduling logic is based on a simple rule: when no tools are currently executing, any tool can start; when tools are executing, a new tool can only start under the condition that "both itself and all currently executing tools are concurrency-safe". Once a non-concurrency-safe tool is encountered, queue processing pauses, waiting for all currently executing tools to complete before the non-concurrency-safe tool runs exclusively.

The hard concurrency cap lives on the static orchestration path: runToolsConcurrently() in toolOrchestration.ts throttles each concurrent batch with getMaxToolUseConcurrency() (default 10, overridable via the CLAUDE_CODE_MAX_TOOL_USE_CONCURRENCY environment variable), preventing file handle exhaustion or I/O contention when the model issues 20 FileReadTool calls at once. The StreamingToolExecutor itself has no numeric cap—it gates purely on concurrency-safety (canExecuteTool()) and can launch arbitrarily many concurrency-safe tools at once (there is no constant named MAX_TOOL_USE_CONCURRENCY).

Additionally, StreamingToolExecutor implements a sibling cancellation mechanism (siblingAbortController): when a Bash tool execution errors out, it cancels other tools executing in the same batch. This avoids the problem of "the first command fails but subsequent commands continue executing." Errors from non-Bash tools (such as FileReadTool, WebFetchTool) do not cascade—their failures are typically independent and don't affect sibling tools.

The emission order of results is always consistent with the order tools appear in the model's output (FIFO), even if later tools finish first. This ensures determinism and predictability of the message stream.