Current limitations

• Same model for summarization: The model specified in your request is used for summarization. There is no option to use a different (for example, cheaper) model for the summary.

• Compaction might fail when tools are defined: When your request includes tools, the model occasionally calls a tool during the internal summarization step instead of writing a summary. When this occurs, the response contains a compaction block with content: null. To prevent this, set instructions to a prompt that explicitly tells the model not to call tools, for example:

  Summarize the transcript inside <summary></summary> tags. Include relevant information in the summary for continuing the task in the next context window. Do not call any tools while writing this summary; respond with text only.