Skip to content

fix: session affinity, claim risk-control, proxy pool health, admission false-kill (with tests) - #6

Merged
7sen1ot merged 10 commits into
7sen1ot:mainfrom
Heking114514:pr/pro-zai-improvements
Nov 9, 2026
Merged

7sen1ot merged 10 commits into
7sen1ot:mainfrom
Heking114514:pr/pro-zai-improvements

Conversation

@Heking114514

Copy link
Copy Markdown
Contributor

Summary

This PR contains 7 groups of fixes identified while running zcode2api in production (35 z.ai accounts, a single long-running Claude Code client, and an outbound proxy pool). All changes are based on v1.6.3 (commit f7c3b8d merged), go test ./internal/server passes, and every fix was verified on a live instance.

1. Session affinity regex fix (bug fix, highest impact)

Claude Code composes metadata.user_id as user_<id>_session_<sid>_account_<acct>, so the session id sits in the middle of the string. The current sessionPattern anchors to $ and can never match, which breaks long-conversation account binding. Replaced with a lazy mid-string match: (?:^|_)session_([a-zA-Z0-9-]{8,128}?)(?:_|$).

Production evidence: after the fix, 576 distinct prompts, 0 cross-account leaks.

2. Claim verification-first flow (risk-control fix)

The old flow sent one ticketless claim to probe for a challenge (HTTP 400), then a second claim with a ticket after SDK verification. Audit logs show this two-request pattern triggers upstream 3012 (duplicate abnormal traffic). Reordered to verification first: automatic claims obtain the ticket before the only claim POST; when verification fails, the upstream claim call count is 0 (locked by a test).

3. Per-claim proxy exit rotation (one card, one IP)

With a manual proxy pool, each card's full flow (scene lookup + SDK verification + claim POST) is pinned to one exit, and the next card rotates to the next exit. Browser-ticket sessions are pinned to their verification-session exit via claimVerificationSession, so ticket binding is never broken. Inference keeps its global sticky exit and is unaffected.

4. Admission timer only guards queueing (false-kill fix)

The o.Admission timer stayed armed until the first token arrived, so it could kill a request that was still doing a cold prefill and label it throttle_window_exhausted (this error is produced by a local cancel; the upstream never returns it). The timer is now stopped as soon as client.Do returns response headers; the in-flight prefill is owned by first_token_timeout.

Production evidence: this error previously accompanied an average of 38.9 attempts per request (max 46) — one request walking 40 accounts and destroying the prompt cache each hop.

5. Account rotation cap (cache protection)

The rate_limit/overload branch never counted toward max_attempts, so one request could walk the entire account pool, shattering the per-account prompt cache across 35 accounts (TTFT p50 27s, hit ratio 0.2). Now:

  • One request may attempt at most 2 accounts (rotationCap)
  • At the cap, untried accounts are never attempted; the request waits out the windows of already-tried accounts (reusing the throttleDeadline mechanism within the total budget)
  • To the client this looks like a slow response, not an error — the gateway absorbs it

6. Proxy pool health: escalating cooldowns, persistence, active probing

  • WAF block cooldown: 1m → 10m base, doubling per repeat (cap 2h), honoring upstream Retry-After; connection-failure cooldowns escalate to 5m
  • New proxy_state table persists block deadlines, so a restart no longer forgets dead exits (previously every restart re-walked dead exits with real user requests)
  • A background prober re-checks all manual exits on startup and every 5 minutes: 401 → release; WAF page → keep blocked 10 more minutes; connection failure → 1-minute quarantine. Dead exits are retired before a user request reaches them

7. Customer API auth and display-name fallback

  • getCustomerInfo sent the raw business JWT without the Bearer scheme and was always rejected with 401; adding the prefix returns 200 immediately
  • Batch-registered accounts have username/customerName/nickName all null upstream; the only readable identity is the masked e-mail shown by the site itself, so its local part is used as the panel display-name fallback

Tests

  • 7 new test files covering: affinity regex forms, claim verification order (claim call count must be 0 when verification fails), exit rotation advance/skip/fallback, admission timer vs in-flight requests, rotation cap, probe verdicts, cooldown escalation math
  • go test ./internal/server passes on the v1.6.3 baseline
  • Verified on a production instance (35 accounts): public inference returns 200, claim audits show the single-request pattern, 35/35 accounts resolve a display name

Notes

  • Per-claim exit rotation currently applies to manual proxy lists (source=manual) only; subscription mode keeps its original behavior
  • The cache_control placement adjustment (system blocks + last message tail only) is intentionally not included; it can be a follow-up if desired

…claim exit rotation

- import-registration endpoint for the registration machine
- claims obtain captcha ticket before the only claim POST (no unverified probe)
- session affinity regex extracts mid-string session ids (user_x_session_y_account_z)
- conversation_affinity_minutes exposed in settings API
- per-claim exit rotation: each card issued from the next pool exit
- upstream fingerprint: fixed win32-x64 client headers, metadata.user_id, VISUALIZATION_ID fill
Release v1.6.3
…ion at 2

- stop the admission timer once the upstream answered (client.Do returned):
  it guards the local queue wait, while first_token_timeout owns the in-flight
  prefill. Kills the spurious throttle_window_exhausted mass-kill loop.
- rate_limit rotations now capped at 2 accounts per request; past the cap the
  throttled accounts re-enter selection so the request waits out their windows
  instead of walking all 35 accounts and shattering per-account prompt cache.
…alth

- WAF block cooldowns now start at 10m, escalate 2x per repeat (cap 2h), and
  honor Retry-After; connection-failure cooldowns escalate to 5m.
- proxy_state table persists block/unavailable deadlines so a restart no longer
  forgets dead exits and re-walks them with user requests.
- background prober re-checks manual exits on startup and every 5 minutes:
  401 releases a stale block, WAF pages park the exit, dead transports get a
  1-minute quarantine. Dead exits are retired before requests hit them.
…ws only

The earlier cap stopped re-queuing throttled accounts but still let fresh
accounts be attempted, so one request could still walk all 35 accounts
(observed: 19 attempts, throttle_window_exhausted at 21:35). At the cap the
scheduler now only re-selects accounts this request already used, and only
after their rate-limit window passes; the request waits out the windows
inside the existing throttleDeadline loop instead of burning the pool.
…mail

The customer info request sent the raw business JWT without the Bearer
scheme, so every lookup failed 401 and accounts showed no official name.
Batch-registered accounts have null username/customerName upstream; the
masked email (e.g. s****g@zlapi.asia) is the only readable identity, so
its local part is now used as the display fallback.
@7sen1ot
7sen1ot merged commit c8006b9 into 7sen1ot:main Nov 9, 2026
1 check passed
7sen1ot added a commit that referenced this pull request Nov 9, 2026
Harden proxy cooldown persistence and probe lifecycle, complete verification-first claims and exit pinning, and retain admission timeout protection. Preserve account-pool rotation without the two-account cap, improve shared busy recovery, and include the existing claim scheduling and runtime fixes.
7sen1ot added a commit that referenced this pull request Nov 9, 2026
Document the integrated fixes and the decision to retain account-pool failover without the two-account cap. Credit @Heking114514 for PR #6 in the changelog and contributor list.

Co-authored-by: hjh <h19037573290@163.com>
7sen1ot added a commit that referenced this pull request Nov 9, 2026
Finalize the approved update notes for v1.6.5 and retain credit for the PR #6 contribution and its integrated improvements.

Co-authored-by: hjh <h19037573290@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants