Repository navigation
feat: default per-stage output caps as shares of the context window - #68
Merged
Merged
Conversation
Until now Node sent no maxOutputTokens unless limits.perCall was set, so one call could write up to the provider's maximum. Each stage now gets a share of compaction.contextWindowTokens between a floor and a ceiling (128k: planner 2000, executor 8000, replanner 1000, synthesizer 4000), like the browser package. - limits.perCall still wins, and 0 means no cap. - A stage that thinks gets no default: reasoning counts against the cap on OpenAI and Gemini, so a default sized for the answer would starve the answer. - A call the cap cuts off is reported: a log warning for the executor and synthesizer, the reason appended to a plan / replanner parse failure. - AGENT_PER_CALL_TOKENS (JSON) sets the caps from the CLI. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011FKop4At26QqqkwVGEjJur
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Node sent no
maxOutputTokensunlesslimits.perCallwas set, so a single call could write up to the provider's maximum, and the browser package behaved differently. Agreed in the thread: derive defaults from the context window, with a floor and a ceiling; an explicitlimits.perCallkeeps winning.What
Defaults — a share of
compaction.contextWindowTokens(default 128k), between a floor and a ceiling, never more than half of a small window:The executor is the widest because a tool call's arguments (a file it writes) count as output. The numbers differ from the browser's (800 / 1,200 / 400 / 3,000) on purpose: Node had no cap at all, so the defaults are generous.
limits.perCallwins;0is an explicit "no cap" (the previous behaviour).A stage that thinks gets no default cap. On OpenAI and Gemini reasoning counts against the output cap, so a default sized for the answer alone could leave a thinking model no room to write it.
A cut-off call says so. Executor / synthesizer emit a
logwarning naming the cap and how to lift it; a plan or replanner decision truncated by the cap fails with the reason appended (same error object, so retry behaviour is unchanged). A default cap otherwise truncates silently.CLI:
AGENT_PER_CALL_TOKENS(JSON, validated eagerly), in--help,env.exampleand the README.Behaviour change
Runs that relied on no per-call cap now get one. Set
perCall.<stage>: 0(orAGENT_PER_CALL_TOKENS={"executor":0}) to restore the old behaviour. Called out in the README.Checks
typecheck,format:check,build,testgreen (409 pass, 1 skipped = live model). Newtests/output-caps.test.tscovers the numbers, windows from 2k to 1M, precedence,0, thinking per stage, the notices, and the real wire (max_tokenson the scripted OpenAI endpoint, incl.finish_reason: length). 21 mutations of the cap logic and 6 of the env parser are all caught. Not run against a live provider.Generated by Claude Code