Skip to content

fix(cloud-agent-next): persist session cost total to stop metadata-read CPU resets - #11571

Open
eshurakov wants to merge 1 commit into
mainfrom
kilo/epic-willow-8w6
Open

eshurakov wants to merge 1 commit into
mainfrom
kilo/epic-willow-8w6

Conversation

@eshurakov

Copy link
Copy Markdown
Contributor

What

Fixes Sentry KILO-CLOUD-AGENT-M: getSessionMetadataSummary / getSession rescanned every
message event with json_extract + sum() on every read, so UI polling against a large
session repeatedly exceeded the Durable Object CPU limit and reset the DO
(operation:getSessionMetadataSummary … "Durable Object exceeded its CPU time limit and was reset.").

The same full scan also ran in getSession and in the v1/v2 history page readers.

How

A single owner for the total — src/control-plane/session/cost-total.ts:

  • Durable running total per (sessionId, kiloSessionId) in a new session_cost_totals table,
    updated in the same transactionSync as every cost-bearing row write (insert / upsert /
    replace / replay / import). Because the total commits atomically with the row, a DO reset can
    never leave it stale against committed rows — no per-read reconciliation scan is needed.
  • Reads are now two single-row lookups (backfill watermark + total) instead of an aggregate
    over the whole message range. getSession, getSessionMetadataSummary, and both history readers
    delegate to the owner; the public response shape is unchanged.
  • Bounded, resumable backfill for sessions that predate the total: an id-keyed cursor advances
    one row window (2048 rows) plus a payload byte budget (4 MiB) per read, persisting progress
    durably. The first read after deploy does not rescan unbounded history; it converges over a
    few reads and survives eviction/reset mid-backfill. Writes made while the backfill runs are
    counted exactly once (rows ahead of the cursor are counted when reached; counted rows apply a
    delta).
  • Exact prior semantics preserved: assistant-role filter, both message.updated (v1 info)
    and conversation.message.updated (native message) shapes, attribution to the payload's own
    sessionID, nonnegative filtering, MAX_SAFE_INTEGER microdollar clamping, and SQLite's
    JSON-boolean→integer cost coercion. A pathological overflowing cost poisons/recovers the sum the
    same way the old scan did.
  • Session wipe (both deleteAll and transactional-locator paths) clears the total and the
    backfill watermark. History replay, delete, and reset recovery are otherwise untouched.

Schema + migration 0007_session_cost_totals (Drizzle DO SQLite, generated artifacts committed).

Verification

Workers-runtime (Miniflare/workerd) integration tests drive the real Session DO and SQLite
storage
— not mocks:

  • New test/integration/control-plane-session-cost-total.test.ts:
    • Large seeded history reads without rescanning rows — asserts a bounded first read and a
      steady-state read of only single-row lookups with no aggregate cost scan. Confirmed to fail
      on the old full-scan behavior
      (reads all ~5000 rows per read) and pass with the fix.
    • Backfill resumes after eviction and counts writes made while it runs.
    • Live writes, replacements, removals, both payload shapes, child-scope isolation, clamping, and
      edge costs stay exactly equal to a verbatim copy of the old full scan (parity reference).
    • Idempotent replay + history import do not double count; both wipe paths clear the tables;
      a failed write transaction rolls the total back.
  • New src/control-plane/session/cost-total.test.ts covers the contribution semantics.
  • Existing suites re-run green: session history, management, core session-v2, native-events,
    transport (replay), rpc, worktree-deletion, scope-handle, and the migration-bundle test.

Checks run (scoped to services/cloud-agent-next):

  • pnpm test — 8007 unit tests pass (325 files).
  • Focused integration suites above — all pass (incl. 233 across the write/replay/delete suites).
  • pnpm typecheck — clean.
  • oxlint on changed src — clean; oxfmt applied to changed files.
  • pnpm check:duplication — no new duplication.

Limitations

  • The integration tests prove the read is bounded and the value is exact in the real Workers
    runtime, but they cannot prove production CPU behavior for the specific large session in the
    Sentry report (no production replay was performed). The fix removes the per-read full scan that
    caused the reset; actual production CPU headroom should be confirmed after deploy.
  • Verification used the Workers-runtime DO integration suite rather than the full fake-LLM
    sandbox/wrapper E2E matrix, which does not exercise this storage path differently.
  • While a pre-existing session's backfill is still converging (a few reads), its reported total is
    the counted-so-far sum and reaches the exact value once the cursor completes; new sessions are
    exact from the first read.

…ad CPU resets

getSessionMetadataSummary/getSession rescanned every message event with
json_extract + sum() on each read, so UI polling against a large session
exceeded the Durable Object CPU limit (Sentry KILO-CLOUD-AGENT-M).

Own the total in one place (cost-total.ts): a durable per-scope running
sum updated in the same transaction as every cost-bearing row write, so a
DO reset can never leave it stale against committed rows. Reads become two
single-row lookups. Existing sessions converge through a bounded, resumable
id-keyed backfill (row window + payload byte budget) instead of an unbounded
first-read scan. Preserve the exact prior semantics: assistant-only, both
message.updated and conversation.message.updated shapes, payload-sessionID
attribution, nonnegative filtering, MAX_SAFE_INTEGER clamping, and SQLite
boolean-cost coercion. Session wipe clears the total and backfill watermark.

Add unit tests for the contribution semantics and Workers-runtime integration
tests: a large seeded history reads without rescanning rows (fails on the old
full scan), backfill resumes after eviction and counts concurrent writes,
replay/import/replacement/removal keep the total exact, and both wipe paths
clear it.
@kilo-code-bot

kilo-code-bot Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: 2 Issues Found | Recommendation: Address before merge

Executive Summary

The durable cost total and its id-keyed backfill reproduce the removed full-scan semantics, but the backfill window still materializes every payload it touches and the read path now performs an unguarded write.

Assumptions: [INFERENCE] Production CPU headroom was not verified (no production replay). Intentional per the PR description: a pre-existing session reports its counted-so-far (lower) total until its backfill converges, and clients display that partial value. Latent, not reachable in normal operation and therefore not commented inline: deleteOlderThan can delete event rows without adjusting the total once it gains a caller, and a poisoned (non-finite) total makes every later cost-bearing write re-run recomputeScope.

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 1
Issue Details (click to expand)

WARNING

File Line Issue
services/cloud-agent-next/src/control-plane/session/cost-total.ts 177 The backfill window select computes length(cast(payload as blob)) for all 2048 returned rows, so the 4 MiB byte budget limits only kilocode message/ rows and large part/ payloads are fully read inside a write transaction on every metadata poll while the backfill is incomplete.

SUGGESTION

File Line Issue
services/cloud-agent-next/src/control-plane/session/cost-total.ts 261 totalCostMicrodollars runs an unguarded backfillStep() write transaction from getSession, getSessionMetadataSummary, and both history page readers, so a storage error now fails a previously read-only metadata read instead of logging and returning the counted-so-far total.
Files Reviewed (11 files)
  • services/cloud-agent-next/src/control-plane/session/cost-total.ts - 2 issues
  • services/cloud-agent-next/src/control-plane/session/cost-total.test.ts - no issues
  • services/cloud-agent-next/src/control-plane/session/event-queries.ts - no issues
  • services/cloud-agent-next/src/control-plane/session/sqlite-schema.ts - no issues
  • services/cloud-agent-next/src/control-plane/session/session-do.ts - no issues
  • services/cloud-agent-next/src/control-plane/session/history-migrations.test.ts - no issues
  • services/cloud-agent-next/src/control-plane/session/drizzle/0007_session_cost_totals.sql - no issues
  • services/cloud-agent-next/src/control-plane/session/drizzle/migrations.js - no issues
  • services/cloud-agent-next/src/control-plane/session/drizzle/meta/_journal.json - no issues
  • services/cloud-agent-next/src/control-plane/session/drizzle/meta/0007_snapshot.json - generated file, no issues
  • services/cloud-agent-next/test/integration/control-plane-session-cost-total.test.ts - no issues

Reviewed by deepseek-flash-latest · Input: 349.7K · Output: 56K · Cached: 6M

Review guidance: REVIEW.md from base branch main

@iscekic iscekic left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bot: Automated code review for this pull request, requested by @eshurakov.

Code Review Summary

Status: 3 Issues Found | Recommendation: Address before merge

Executive Summary

The durable (session_id, kilo_session_id) cost total with an id-keyed resumable backfill reproduces the removed sum(json_extract(...)) scan faithfully; the remaining risk is on the backfill read path, where the per-read byte budget only counts message rows and the metadata poll now runs an unguarded write transaction.

Assumptions: [INFERENCE] No production replay was performed, so the CPU headroom claim for the specific large session is unverified. The partial total reported while a pre-existing session's backfill converges is intentional per the PR description; for a very large session that is one 2048-row window per read, so the displayed total can stay low across many polls before converging.

Overview

Severity Count
CRITICAL 0
WARNING 2
SUGGESTION 1
Issue Details (click to expand)

WARNING

File Line Issue
services/cloud-agent-next/src/control-plane/session/cost-total.ts 194 The 4 MiB payload budget accumulates only message/ candidates, so the window is bounded by 2048 rows regardless of size; a heavy window that cannot finish rolls back and is retried forever from the same cursor.
services/cloud-agent-next/src/control-plane/session/cost-total.ts 261 totalCostMicrodollars runs an unguarded backfillStep() write from getSession, getSessionMetadataSummary and both page readers, so a failing step fails a read that previously only flushed part text through error-isolated onFlushError.

SUGGESTION

File Line Issue
services/cloud-agent-next/test/integration/control-plane-session-cost-total.test.ts 225 The byte-budget break and its cutoff handling are never exercised by the seeded history, because every seeded payload is small and the first window is all message rows.

What I checked

  • Semantics parity with the removed scan. messageCostContribution matches the old SQL predicate for predicate: kilocode rows only, the message/…message0 range (equivalent to startsWith('message/') under BINARY collation), both message.updated/info and conversation.message.updated/message shapes, assistant role only, attribution to the payload's own sessionID, and the same typeof cost in ('integer','real') → max(0, cost) coercion (including JSON booleans contributing one dollar and non-numeric costs contributing zero). Overflow still yields Infinity, which totalCostMicrodollars reports as 0 exactly as the old Number.isFinite guard did.
  • The incremental total. Every production write to events goes through the control-plane write() wrapper (insert, upsert, insertUnique), including native events, kilo2 conversation events and the part-text-buffer flush, and message.removed maps to the same message/<id> entity so the before/after delta subtracts the removed cost. Raw inserts into events exist nowhere in src/, so nothing bypasses the total after the backfill completes. Rows written ahead of the cursor are skipped and counted once when the window reaches them; sessionCostTotals and costBackfillState are cleared in both wipe paths, and deleteAll re-runs migrate so the new tables exist again.
  • Overflow recovery. A poisoned non-finite total is replaced by recomputeScope on the next cost-bearing write, scoped to the cursor while the backfill is running, so it does not double count the uncounted range.
  • Migration artifacts. 0007_session_cost_totals.sql, migrations.js, _journal.json and the 0007 snapshot are consistent, the snapshot prevId chains from 0006, and the schema defaults match the generated SQL.
  • Tests. The new suites would fail without the change; the integration cases cover bounding, eviction resume, write-while-backfilling, both payload shapes, child-scope isolation, clamping, removal, replay/history import, failed-transaction rollback and both wipe paths.

Not verified here

Nothing is installed in this checkout, so I ran no test, typecheck or lint. The Workers-runtime suite in test/integration/control-plane-session-cost-total.test.ts is the right place to prove the backfill behaviour, and it was not executed by this review.

Act on first

Line 194: count row.bytes for every row in the backfill window, not just message/ candidates, so one metadata poll cannot pull an arbitrary payload volume — and so a window that cannot finish cannot wedge the cursor.

E2E: not run.

No end-to-end or integration check was executed. Nothing is installed in this checkout and this job installs no dependencies, so the Workers-runtime suite (test/integration/control-plane-session-cost-total.test.ts), the unit suite and typecheck could not be run here. I reviewed the change statically against the removed SQL and the call paths. For context only, the checks reported at head 9b1704a show verify and format-check as SUCCESS; that is the PR's own CI record, not a run performed by this review.

Every remark is attached to the changed line where GitHub accepts one.

This service never pushes, rebases, forces, closes, labels, or merges, and it posts nothing else on this pull request.

for (const row of window) {
if (row.stream_event_type === 'kilocode' && row.entity_id?.startsWith('message/')) {
if (bytes + row.bytes > COST_BACKFILL_BYTE_BUDGET && cutoff !== state.cursor) break;
bytes += row.bytes;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: The 4 MiB payload byte budget only counts message/ rows, so a backfill step is bounded by row count alone, not by bytes.

bytes is incremented only inside the candidate branch (lines 192-196), while the window select at lines 172-183 still measures and reads length(cast(payload as blob)) for every row it returns. part/ rows are the bulk of a large session's bytes and are never candidates, so a window of 2048 rows can touch far more than COST_BACKFILL_BYTE_BUDGET on a single metadata poll. That matters because backfillStep() is invoked from getSession / getSessionMetadataSummary (session-do.ts:2875 readSessionSummary) and from both history page readers, i.e. the exact polling path this PR exists to make cheap.

The failure mode is not just a slow read: the step runs inside storage.transactionSync. If one step cannot finish (CPU limit, storage error), the transaction rolls back, cost_backfill_state.cursor never advances, and the next poll retries the same window. A session whose window is too heavy therefore never converges.

Smallest fix: accumulate row.bytes for every row in the window (before the candidate check) and break on the budget, keeping the existing "always take at least one row" rule so the cursor still strictly advances; then the per-read payload bound matches the 4 MiB the PR describes.

addDollars(afterContribution, afterContribution.dollars, state);
},
totalCostMicrodollars(sessionId: string, kiloSessionId: string): number {
backfillStep();

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: A previously read-only metadata read now performs an unguarded write transaction.

totalCostMicrodollars calls backfillStep() (line 167), which opens storage.transactionSync and writes session_cost_totals + cost_backfill_state. The callers are getSession, getSessionMetadataSummary and both history page readers (history.ts:180, conversation-history.ts:100). While the backfill is incomplete, any throw out of that transaction (storage failure, a step over the DO's CPU budget) propagates and fails the whole read — the caller gets no snapshot at all instead of the counted-so-far total.

The file already has the right precedent for a write driven by a read: readSessionSummary flushes buffered part text, and createPartTextBuffer takes onFlushError (session-do.ts:849-855) so a failed flush is logged and the read still returns.

Smallest fix: wrap the backfillStep() call in a try/catch, log at error level with the session id, and continue to readDollars(...); the next read retries the step from the unchanged cursor. That keeps the metadata read serving the explicit partial total the PR documents instead of turning it into a hard failure.

it('reads large stored history through a bounded backfill, then without rescanning rows', async () => {
const { stub, sessionId, kiloSessionId } = await fixture();
const assistants = 5000;
await seedStoredHistory(stub, sessionId, kiloSessionId, assistants, 1000);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: The byte-budget break in backfillStep is the one branch with no test, and it is the branch that can silently lose cost.

Every seeded payload here is small (to a 2048-byte part at line 136, ~250-byte assistant payloads), and the first window is entirely message/ rows, so COST_BACKFILL_BYTE_BUDGET is never reached and the break at cost-total.ts:193 never runs. That branch decides cutoff, and cutoff decides both which row the next step resumes from and which id range recomputeScope counts. A one-row-off mistake there drops a message's cost with no other test noticing, because the parity assertions all run after the backfill is complete.

A case worth adding: seed a few byte-heavy non-message rows (or one assistant payload above the 4 MiB budget) ahead of the assistant rows, then assert (a) the first step's cursor stops before the un-counted row, (b) the total after convergence still equals referenceTotalMicrodollars. That is the missing case the change's own risk demands, and it also pins down the behaviour I raised on line 194.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants