Repository navigation
Recover when a pinned OpenRouter upstream refuses a call - #318
Merged
Merged
Conversation
Cache learning can skip every upstream of a model but one. If that last upstream then refuses a call (GMICloud answered tencent/hy3 with a 400 unsupported_operation), OpenRouter has nowhere to fall back and the run fails. When a call our routing shaped fails at a named upstream, unpin the bot, lift the model's cache skips, skip the refusing upstream instead (unless it is the only one), and retry the call once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A run on
tencent/hy3failed with400 Provider returned error(unsupported_operation, provider GMICloud). The model has three OpenRouter upstreams. Cache learning had skipped Tencent and Novita and kept GMICloud only because it was the last one. When GMICloud refused the call, OpenRouter had no other upstream to fall back to.ModelRetryMiddlewaredoesn't retry a 400, so the run failed.Fix
failed_upstream()reads which upstream OpenRouter blamed (error.metadata.provider_name), following langchain's wrapped exception back to the original.StickyUpstreamMiddlewarehandles a call that our pin or skip list shaped and that failed at a named upstream. It unpins the bot, lifts the model's cache skips, skips the failing upstream for 6h (unless it's the only one), and retries once.openrouter_provider_orderis set, the error is raised as before.Tests
tests/test_upstream.py. Two of them fail without the fix.🤖 Generated with Claude Code