Repository navigation
Conversation
An endpoint that accepts only tool_choice "auto" refuses the named submit and then "required" on every forced submit, so each one sent two refused requests before the extractor answered. Keep those refusals per client for the life of the node; a later forced submit on that client starts at the first choice its provider has not refused, or goes straight to the extractor.
Owner
Author
|
Closing without merging. The cost this removes is two refused requests per forced submit on an endpoint that accepts only "auto". The provider rejects them before running the model, and the extractor answers on the third request. Keeping a node-wide record of refusals is more state than that cost justifies. The tests pinning the current fallback order stay as they are. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What happens now
When a ReActV2 turn ends with a forced
submit, Imp first namessubmitintool_choice. If the provider refuses that, Imp asks again withtool_choice: "required"andsubmitas the only tool. If the provider refuses that as well, the tools-disabled extractor writes the answer.An endpoint that accepts only
"auto"(Meta's API, relayed by OpenRouter: "only\"auto\"is supported fortool_choice.\"none\",\"required\", and named function choices are not currently supported") refuses both requests on every forcedsubmit. So every forcedsubmiton such a model sends two requests that are certain to be refused before the extractor's request succeeds.test/react_v2_resume_extract_test.exsalready pins that sequence for one resume.Change
The provider's refusal is kept for the client that received it, for the life of the node (
:persistent_term, written once per client and choice). A later forcedsubmiton the same client starts at the first choice the provider has not refused:"required"both refused: the request goes straight to the extractor, so the first request sent is one the provider accepts"required"is sent directlyThe model registry can't answer this question. LLMDB records whether a model takes tools, and OpenRouter's
supported_parametersliststool_choicefor an endpoint that accepts only"auto", so the refusal is the only evidence available. The memory is keyed by the client's module, its model spec and a hash of the whole client. Two clients for the same model that differ in endpoint, options or transport are therefore remembered separately, and the client's API key is never stored. A client with no model spec, such as a static or scripted LM, is not remembered. A provider that starts accepting a forced choice will be forced again after a restart.Tests
In
test/react_v2_resume_extract_test.exs, each test runs a wire-level local HTTP server that returns the real refusal body:refuses_both: a second resume on the same client sends only the extractor's request. Before this change it failed and sent named, then required, then the extractor.refuses_named: a second resume sends only"required". This also failed before the change.:forced_submit.test/react_v2_*.exs,react_contract_test.exsandrun_request_record_test.exspass under five seeds. Those runs check that the existing fallback tests, which share a model spec but use different stub clients, don't affect each other.