Skip to content

Recover when a pinned OpenRouter upstream refuses a call - #318

Merged
regnull merged 1 commit into
mainfrom
fix/upstream-refusal-fallback
Nov 18, 2026
Merged

regnull merged 1 commit into
mainfrom
fix/upstream-refusal-fallback

Conversation

@regnull

@regnull regnull commented Nov 18, 2026

Copy link
Copy Markdown
Owner

Problem

A run on tencent/hy3 failed with 400 Provider returned error (unsupported_operation, provider GMICloud). The model has three OpenRouter upstreams. Cache learning had skipped Tencent and Novita and kept GMICloud only because it was the last one. When GMICloud refused the call, OpenRouter had no other upstream to fall back to. ModelRetryMiddleware doesn't retry a 400, so the run failed.

Fix

  • failed_upstream() reads which upstream OpenRouter blamed (error.metadata.provider_name), following langchain's wrapped exception back to the original.
  • StickyUpstreamMiddleware handles a call that our pin or skip list shaped and that failed at a named upstream. It unpins the bot, lifts the model's cache skips, skips the failing upstream for 6h (unless it's the only one), and retries once.
  • If OpenRouter routed the call itself, or openrouter_provider_order is set, the error is raised as before.

Tests

  • Three new tests in tests/test_upstream.py. Two of them fail without the fix.
  • The full backend suite passes (864 tests). Ruff is clean.

🤖 Generated with Claude Code

Cache learning can skip every upstream of a model but one. If that last
upstream then refuses a call (GMICloud answered tencent/hy3 with a 400
unsupported_operation), OpenRouter has nowhere to fall back and the run
fails. When a call our routing shaped fails at a named upstream, unpin the
bot, lift the model's cache skips, skip the refusing upstream instead
(unless it is the only one), and retry the call once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@regnull
regnull merged commit 73bad03 into main Nov 18, 2026
4 checks passed
@regnull
regnull deleted the fix/upstream-refusal-fallback branch November 18, 2026 01:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant