Skip to content

OpenRouter: end a chat answer with its web-search sources - #49

Merged
robdevops merged 31 commits into
mainfrom
claude/compassionate-heisenberg-pz23nq
Nov 5, 2026
Merged

robdevops merged 31 commits into
mainfrom
claude/compassionate-heisenberg-pz23nq

Conversation

@robdevops

Copy link
Copy Markdown
Owner

OpenRouter's web search returns the pages it used as url_citation annotations, but the backend only counted them, so replies cited a source only when the model chose to.

  • openrouter.py keeps the citations (http(s) only, one per URL) on Step.sources; run_search (the :online model path) also hands the model the links.
  • runner.run puts the sources of the last round that cited any on Answer.sources.
  • ask() appends Sources: <a>…</a> · … (max SOURCES_MAX = 3) to a chat answer that has no link of its own. Holding news and filings are left alone because they parse their own text.
  • Not done: inline per-sentence links (the annotation offsets don't survive the formatting pipeline) and the xAI path.
  • Also includes the import-order fix ruff was flagging in charts/render.py, and a merge of main (the branch still held the commits squash-merged in Charts: draw ² and other missing glyphs as plain stand-ins #48).

No live OpenRouter in the sandbox: the annotation shape follows OpenRouter's documented response and is covered by fakes only.


Generated by Claude Code

claude added 30 commits November 2, 2026 23:43
… plain stand-ins

Pillow's default font has no ² ³ − → € etc., so they drew as boxes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
…ws the share)

Falls back to shares when a value would round to zero.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
…hes value legend

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
…ks none

The url_citation annotations were only counted; now they are kept on the Step and Answer
and ask() appends a Sources line (max 3) to chat answers that carry no link of their own.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
Positions from the url_citation annotations are applied to the raw reply before any
formatting; the Sources line stays as the fallback when no marker could be placed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
…CHAT; log user input as two words

- LOG_WIDTH removed: tool-name lists are logged in full.
- BOT_TZ -> TZ, HISTORY_LIMIT -> HISTORY_MAX (default 10, and a real maximum: the
  stepped window now holds max/2+1 .. max messages), TELEGRAM_PRELOAD_CHAT ->
  TELEGRAM_IMAGE_PRELOAD_CHATID. The old names are not read any more.
- User input in the log is cut to its first two words and one ellipsis character; the
  other log-line truncations (error snippets, failed HTML) are gone.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
…Default column

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
…rs were placed

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
OpenRouter's url_citation positions are 0 and the search runs in an earlier round than the
answer, so the position-based marker is replaced by matching the answer's sentences to each
source's snippet (figures weigh most, words every snippet shares count for less).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
The chat system prompt (OpenRouter only) tells the model to cite a sentence that rests on a search
result as [<a href=URL>n</a>]. Its links are kept only for pages the search returned and renumbered;
without them the snippet-overlap markers apply. The trailing Sources line is gone.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
…ere dropped

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
…n order

The model cited 3 of ~10 pages the search returned; the overlap pass now runs for the rest even when
the model wrote markers of its own.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
Bare capitalised words (GAAP, EBITDA, AFR, GM, UPS) kept getting Yahoo links and the stoplist never
ended; ASX:NXT linked to the US NXT. Replies now link only $NVDA, NXT.AX, ASX:NXT (mapped to
Yahoo's suffix), Name (TICKER), Sharesight's NXT (ASX), ^GSPC and BTC-USD, plus bare words in a
known set: the S&P 500 and NASDAQ-100 (from Wikipedia) and the US Sharesight holdings, refreshed
daily in the background and kept in kv, never fetched while answering. A bare code in brackets
links only if known, since it names no exchange. Holding news links the holdings' codes and a
movers explanation the listed codes with the list's exchange. The prompt asks for these forms.
The NOT_TICKERS env var is gone; the built-in list now only serves the tool gate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
- An exchange suffix links on its own: ALL.AX (Aristocrat), ASX.AX, AI.PA were left plain because
  the code is also on the acronym list.
- A bracket that isn't a ticker ("NVDA (Q3 FY26)") no longer swallows the ticker before it.
- Share classes: BRK.B as the model writes it links to Yahoo's BRK-B.
- "AT&amp;T (T)" and "Procter &amp; Gamble (PG)" are bolded whole.
- Numeric holding codes (700 on HKEX) never link bare numbers, and keep their unsubscribe button.
- Index tickers that are also finance or tech acronyms (GM, UPS, IR, HD, USB, RF...) link only in a
  marked form, as single letters already did.
- A refresh part that fails (a page down, Sharesight not up yet) is tried again in 30 minutes,
  not 24 hours; a footnote in the table header no longer hides the ticker column.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
…the ASX 200

Wikipedia answered 403 to the page fetches: its policy refuses programs whose User-Agent names no
contact. The lists now come through the action API (action=parse) with the repo URL in the
User-Agent, and a refusal logs Wikimedia's reason. The S&P/ASX 200 is added: its codes link bare
with .AX. Holdings on any known exchange count (NXT -> NXT.AX). A code in a US index and in the
ASX 200 isn't linked bare unless a holding settles it. ASX codes that are everyday words or
acronyms (GPT, WOW, ZIP, CAR...) need a marked form.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
The "Nasdaq-100" article has no constituents table any more (0 tickers in production).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
…gle Finance

Google takes CODE:EXCHANGE (NXT.AX -> NXT:ASX, 0700.HK -> 700:HKG); a US ticker names no exchange
(NASDAQ or NYSE), so it opens a Google Finance search. History compaction and the holding-news
mute codes read either provider's links back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
Google Finance quotes are CODE:EXCHANGE (JPM:NYSE, SPCX:NASDAQ, BRK.A:NYSE). A US ticker's exchange
now comes from the text (NYSE:JPM, JPM (NYSE)), the Sharesight holdings, the NASDAQ-100 (all
NASDAQ) or the S&P 500 list, whose ticker links point at nyse.com or nasdaq.com; kept in kv with
the lists. Only a ticker none of these place opens a Google Finance search.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
TradingView pages are EXCHANGE-CODE (ASX-NXT, NYSE-BRK.A, NASDAQ-SPCX), with the US exchange from
the same sources as Google's; a code it can't place opens TradingView's page for the bare code.
The three sites' URL builders leave tickers.py (near its size limit) for their own module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
Shows what the daily holding-news check (and movers, DM buttons, chat commands, filings) costs
next to chat; every request was already recorded with its kind.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
/tokens and /credits still work. Telegram shows the menu commands in autocomplete, so admins now
see /usage there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
Each recipient used to get a separate searched request, so a stock two people hold was searched
twice and every request paid its own overhead (holding news was the biggest cost in /usage).
The daily check now asks once for the union of everyone's unmuted holdings, with everyone's recent
items as the don't-repeat list, then gives each person the items about their own holdings; an item
naming no holding goes to everyone. /holdingnews still checks just the asker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
… articles

SHARESIGHT_HOLDING_NEWS_SKIP_DAYS (e.g. sun,mon) skips the daily check on those weekdays. Each
check now searches back to the last one that got an answer (kv; at least 24 h, at most 7 days),
so skipped days, a busy provider or an outage leave no gap; the prompt and the message header
say how far (past 72h). Repeats: the don't-repeat list always reaches past the last check, and
an item citing an article already sent is dropped whatever the model says. The feature becomes a
package (holding_news/ with text.py) to stay under the module size limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
The old name is no longer read: rename it in the service's environment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014k4kywyN5dAo39izUPvxbw
@robdevops
robdevops merged commit 3092986 into main Nov 5, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants