Skip to content

feat(runtime): add Gemini, OpenAI pro, and long-context pricing tiers - #3389

Open
Mattbusel wants to merge 1 commit into
ultraworkers:mainfrom
Mattbusel:use-llm-prices
Open

Mattbusel wants to merge 1 commit into
ultraworkers:mainfrom
Mattbusel:use-llm-prices

Conversation

@Mattbusel

Copy link
Copy Markdown

Your pricing table returns None for every gemini-* id, so a Gemini model routed through an OpenAI-compatible gateway gets the Sonnet default ($3/$15) and the "pricing=estimated-default" tag. o1-pro, o3-pro, gpt-5-pro, gpt-5.2-pro, gpt-5.4-pro, gpt-5.5-pro and gpt-5.3-codex also return None. For gpt-5.5-pro that means $3/$15 instead of $30/$180. The long-context tiers your comments call unmodeled (Haiku 5.5 over 100K at 5x, OpenAI over 272K, Gemini 2.5/3.1 Pro over 200K) are missing too, even though Usage::estimated_cost_usd in the api crate prices one response at a time and knows that response's prompt size.

This PR adds those rows to OPENAI_COMPATIBLE_PRICING, using your id matching and conventions. It also adds a small LONG_CONTEXT_PRICING table and a pricing_for_request(model, prompt_tokens) that returns the higher tier once a single request crosses the threshold, and Usage::estimated_cost_usd now calls it. pricing_for_model(), the session and cumulative totals, and your Anthropic, DeepSeek, xAI, Qwen and Kimi rows are unchanged. Tests cover the new rows and both sides of each threshold. I took the numbers from llm-prices, a crate I wrote that keeps dated, source-cited LLM prices (table checked 2026-10-23). CONTRIBUTING asks to avoid new dependencies, so this copies the rows and adds no dependency. If you'd rather pull them from the crate behind an optional feature, or leave out the tier part and keep only the rows, I can change it.

Adds Gemini API rows and the missing o1-pro, o3-pro, gpt-5.x-pro and
gpt-5.3-codex rows to OPENAI_COMPATIBLE_PRICING, and a per-request
pricing_for_request() that applies the Haiku 5.5 >100K, OpenAI >272K and
Gemini >200K prompt tiers. Usage::estimated_cost_usd uses it, since it
prices one response. pricing_for_model() and aggregated session totals are
unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant