[kixelated]

Finishes quest/m0/benchmark/time-split.md: every session record now says where its working time and tokens went beyond its timed waits, as a cheap model sorted its steps, with a line on what went wrong and a measured classifier error.

Summary

• SessionRecord.split (src/core/record.ts): per category {ms, tokens} over a fixed list (understanding, researching, writing, testing, environment, git, forge, reviewing, rework, coordinating, operating, reporting, circles, waiting, other, plus the measured idle), one-line notes, a check (time a stronger model labeled again, and the part it labeled differently), and usage (what labeling took, by model). parseRecord validates it whole. The list carries a dated comment, since changing it breaks the trend.
• src/cli/split.ts: reads a Claude Code or Codex transcript into numbered actions (each call or message, trimmed, with its result trimmed), asks the model to label them as ranges, and computes time and tokens from the transcript itself. Each stretch goes to the action it ends at, a turn's message ends an idle stretch, and timed waits are subtracted. The categories plus the waits therefore sum to the span exactly, and the tokens to the transcript's, whatever the model judges. Labels are kept under the sender's state directory, so each stop labels only the new actions. A digest of the labeled actions detects a transcript that reads differently and labels it afresh.
• Sender (send.ts): labels each emitted local session before building its records. Labeling never holds a record back. A session whose transcript can't be read, or that no model could label, is sent with split: null and the queue settles; the next stop says why, and the session's next stop labels the parts still unlabeled.
• Fallbacks: ask takes a parse, so an unusable reply (not JSON, no session, or labels that don't cover the part) passes the question to the next model. A check no model could give leaves its part unchecked and keeps the splitter's labels. What unusable replies took still counts in split.usage, since ask throws UnansweredError with every turn that replied. A note past NOTE_LIMIT is cut, not refused.
• Models (models.ts): SPLITTERS (Haiku low, else Sol low) and SPLIT_CHECKERS (Sonnet medium, else Sol medium), so the model check retunes them.
• review.ts: the risk classifier's one-turn loop became a shared ask(), which the splitter reuses. A question with tools: "" runs Claude with --tools "" and Codex with shell, exec, multi-agent, image, browser, computer-use, goals, view-image, and web search off.
• quest usage: a per-PR time: line and went wrong notes, plus a Time by role rollup with disagreement and labeling tokens. It reads only kept labels and never asks a model.
• README: a new "Time splits" section; the record and usage docs now cover the split.

Security finding fixed on the way

Under quest review's existing flags, Codex still exposed the user's connected ChatGPT apps (mail, a GitHub app with merge and write tools, Drive, and others) with approval_policy="never"; the read-only sandbox does not cover them. A diff under review or a transcript being classified could ask for them. Every Codex turn quest review runs now passes features.apps=false. Reviewers and the risk classifier keep their shell, which they use to read the diff. Verified by asking codex exec with these flags to list its tools and to read /etc/hostname: apps and web were gone, and it answered NOACCESS.

Choices made without asking

• The model labels steps; it does not return per-category time. The Plan asked for time and tokens per category summing to the totals. Labels with code-computed sums meet that exactly, where a model's own arithmetic would not. Rejected: having the model emit numbers and normalizing them.
• Category list, settled from about 50 sampled transcripts on this host (Claude Code and Codex, implementing, iterating, and coordinating sessions). In that sample, tool runtime was a minority of the time; most went to model turns between calls, which is why the categories describe intent ("why it was done"), not the tool. I added researching, environment, coordinating, operating, reporting, and waiting beyond the Plan's examples, because the sample spent real time there. idle is measured, never judged.
• Labeling runs at every stop, on the new actions only. A busy coordinator pays one small call per stop (about 25k input tokens of prompt per part measured below), and quest usage shows the total. Rejected: re-labeling the whole transcript each stop, which is quadratic, and waiting for a session to go quiet, which the queue has no signal for.
• Its own cost is tokens in split.usage, not a separate priced record. The classifier's claude -p/codex exec sessions run from the state directory, so no PR claims them. quest prices nothing itself, and emitting them as their own records would need new lineage plumbing. The report shows what measuring takes, by model.
• Codex transcripts are labeled by Haiku too, so their trimmed step text reaches Anthropic's API when Haiku runs. Only categories and notes leave for the store.
• Sampling: about one part in ten, chosen by a hash of session and part, so a resend never re-samples differently.
• Disagreement is time-weighted: the share of checked time the two models labeled differently.
• The note is a single model-written line of at most 300 characters, told to name no one and quote no message. parseRecord refuses a multi-line or longer note. Nothing can verify mechanically that it quotes nothing.
• The plan's "Haiku/Jev": used claude-haiku-5-5, as briefed.

Public API and wire impact

• Wire (breaking): SessionRecord gains a required split field (TimeSplit | null), and parseRecord refuses a record without it. A store parsing with an older quest/core refuses new uploads as having an unknown field split. To migrate, the store updates quest/core. Nothing stores records in production yet (records-endpoint still waits on the store).
• quest/core exports SPLIT_CATEGORIES, SplitCategory, SplitUsage, TimeSplit, and NOTE_LIMIT.
• quest usage and --json gain the split lines and a splits array. quest records sender now asks models.

Validation

• just check and just test pass: 1360 tests, 5 skipped, after merging feat(records)!: record what each merged PR broke #726. New test/split.test.ts covers Claude and Codex parsing (usage deduped per reply, Codex running totals), label validation (gaps, overlaps, out-of-range, idle, long notes), the split's sums with overlapping waits, incremental labeling, relabeling after the transcript changes, checker sampling, and parseRecord's split rules. New tests in send, usage, and review cover the upload carrying the split, a labeling failure keeping the queue, the report lines, and the no-tools flags.
• Real transcripts: parsed six of this host's sessions (12 actions up to 1,785, spans up to 12 hours) in at most 57 ms each. Their token sums matched an independent Python reader exactly on all six.
• Real models: labeled two real sessions end to end through ask and labelSession. Each category's time summed exactly to the transcript span, and each one's tokens to the transcript total. Each part cost about 25k input and 0.5 to 0.8k output tokens. The notes came back generic, quoting nothing.
• Outside a checkout: after review round 1, labeled a real Codex transcript from a state directory outside any Git repository, which the sender's state directory is. Codex had refused that before --skip-git-repo-check.
• Classifier error: labeled the first part of six more real sessions with the splitter and the checker side by side. They disagreed on 16% of about 350 checked minutes. The largest confusions were rework/waiting and understanding/testing.
• Not exercised here: Haiku and Sonnet. This host's claude reports "Not logged in" (it routes through ANTHROPIC_BASE_URL without a credential in the environment), so every real run above fell through to gpt-6.1-sol low and medium, exactly as the ranked lists intend. The 16% therefore compares Sol low with Sol medium, not Haiku with Sonnet. token-max is not installed on this host either, so the sender itself ran only in tests.

Caused by

No single PR: the time split is new work. The Codex apps exposure is as old as Codex reviewers in quest review; it surfaced because Codex added connected apps, not through a change here.

Follow-ups

• Give this host's Claude CLI a login, so Haiku and Sonnet can label and check here. Recommend: drop here; it is host configuration, which lives in kixel-ai/dotfiles.
• None otherwise: the first week of Time by role shows whether the category list or the per-stop cost needs a change.

Decisions

• Round-3 design checkpoint (maintainer, feat(records)!: split each session's time into categories with a cheap model #725 (comment)): ✅ Continue: carry failed tries' tokens on ask's error and record them in split.usage (rejected: label after sending, as a separate pass; price labeling through token-max as records of its own). Done: UnansweredError carries every turn that replied, and labelSession adds them for a skipped check and for a part left unlabeled.

Models

• Size: M
• Implementer: Claude Code, claude-opus-5-5, medium effort
• Implementer: Claude Code, claude-opus-5-5, medium effort (iterate, M)

🤖 Generated with Claude Code

(Written by Claude Opus 5.5 with medium)

Review

Rewritten by quest review after each round.

• Head: 1d7a30b

• Base: origin/main at 05e59e9

• Risk: high, The required split field changes record parsing for downstream consumers, automatic transcript labeling can spend money and disclose transcript data to models, and reviewer tool restrictions change a security boundary; reverting cannot undo spending or disclosure.

• Classified: high by Codex, gpt-6.1-sol, low effort, session 01a1d81f-2a11-7930-9f52-30c7a1421bb0

• Reviewer: Codex, gpt-6.1-sol, high effort

• Session: 01a1d7e9-2d4c-7312-93bb-0600b47f5393, round 4

• Score: 10/10, The earlier accounting finding is fixed, and reading the incremental changes and merge integration revealed no new issues.

• Rounds: 4; first round 3 (high 1, medium 2); findings 4 (high 1, medium 3): fixed 4, refuted 0, open 0

• Findings: none

• The last round's findings, at f41c370:

  • Fixed: medium, src/cli/split.ts:508: Preserve paid checker usage when every fallback fails; UnansweredError now carries failed replies' usage, which src/cli/split.ts:524 adds before saving retained labels at line 530; lines 504–511 also persist failed splitter usage for later attempts.

• Reviewer: Codex, gpt-6.1-sol, medium effort, shadow

• Session: 01a1d7e9-2d4c-7ed0-9934-75e1feaff2a7

• Rounds: 2; first round 2 (high 1, medium 1); findings 2 (high 1, medium 1): fixed 2, refuted 0, open 0

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 1 minute.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 98e58f61-1563-4a51-99c0-e719a84fc02d

Commits

Reviewing files that changed from the base of the PR and between 05e59e9 and 1d7a30b.

Files selected for processing: 16

• README.md
• quest/m0/benchmark/time-split.md
• src/cli/main.ts
• src/cli/models.ts
• src/cli/records.ts
• src/cli/review.ts
• src/cli/send.ts
• src/cli/split.ts
• src/cli/usage.ts
• src/core/index.ts
• src/core/record.ts
• test/records.test.ts
• test/review.test.ts
• test/send.test.ts
• test/split.test.ts
• test/usage.test.ts

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[pullfrog]

ℹ️ No critical issues — three rough edges in how labeling failures are handled, inline.

Reviewed changes

I reviewed the full PR at 2c8b5dd: the new time split and the record, sender, usage, and review changes that support it.

• SessionRecord.split: a required TimeSplit | null with fixed SPLIT_CATEGORIES. parseSplit checks every field, and a split is refused when waits is null.
• src/cli/split.ts: reads Claude Code and Codex transcripts into actions, marks, and turns. A model labels each action, and labels are kept per session behind a digest so each stop labels only new actions. splitOf computes time and tokens per category from the transcript, after subtracting merged waits and clipping to since.
• Sender: labels each local session it emits before building the records. A labeling failure goes through the group's existing backoff.
• ask() refactor: the classifier's fallback loop is now shared. A question with tools: "" runs Claude with --tools "" and Codex with its tools off and --skip-git-repo-check.
• Codex apps off: every Codex run now passes features.apps=false, which closes the exposure of connected apps described in the PR body.
• quest usage: adds a per-PR time: line, went wrong notes, a Time by role rollup, and a splits array in --json. All of it is read from kept labels.

ℹ️ The Claude labeling path has never been run against a real model

The PR body says every real run fell through to Sol, because this host's Claude CLI is not logged in. So claude -p --tools "" --json-schema … with Haiku and Sonnet has never answered a labeling question. Community reports about toolless --json-schema are mixed. Some say it works, and one says toolless sessions sometimes return placeholder keys. On a host where Haiku is logged in, a bad answer would not fall through to Sol, as the first inline comment explains. It would fail every pass instead. One real Haiku labeling run before merge would settle this.

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using claude-opus-5-5 ｜ 𝕏}

[moq-bot]

quest land watch: ended this pull request's pick, disarming its auto-merge, since its head 2c8b5dd is held by finding by pullfrog[bot]: A labeling answer that arrives but is unusable fails the whole repository group's upload, and the next model is never tr (#725 (comment)); finding by pullfrog[bot]: When the checker fails, the splitter's labels are thrown away even though they were valid and already paid for. If the c (#725 (comment)); finding by pullfrog[bot]: A note that runs past NOTE_LIMIT fails the whole part. Its paid labels are discarded, and the group's upload fails wit (#725 (comment)). Push a fix, or answer each by a thread reply or a comment linking its review or decision, then land it again.

(written by quest land watch)

[pullfrog]

✅ No new issues found. The three earlier findings are fixed, and the fixes add no new problems.

Reviewed changes

Since the review at 2c8b5dd, one commit changes how labeling failures are handled. It moves reply validation inside the model fallback and makes labeling failures non-blocking.

• ask<T> validates inside the fallback: Question.parse runs on each reply. A missing session, a reply that is not JSON, or a parse throw now becomes a ReviewInputError, so the question passes to the next model. Answer.spent keeps every turn that replied. The risk classifier uses the same path, so a malformed classifier reply now exits through unanswered.
• The splitter parses in ask: parseLabels runs as the question's parse. A note past NOTE_LIMIT is cut by oneLine and no longer fails the part. addUsage sums spent, so unusable replies are counted whenever some model answered.
• A failed check is skipped: if no checker can answer, with ReviewInputError or ReviewLimitError, the part stays unchecked and the splitter's paid labels are still written. Any other error still propagates.
• The sender never holds a record back: any labeling error except ENOENT becomes a per-session note. The record goes out with split: null, the group settles, and the next stop reports why. Labels are still written after each part, so the parts already labeled are kept.
• Tests and README: the tests cover classifier fallthrough on unusable replies, checker failure keeping labels, a cut note that parseRecord accepts, and an unsplit send followed by labeling on the next stop. The README and --help text describe the new behavior.

The Claude labeling path is still untested against real Haiku and Sonnet. The risk is lower now, because an unusable Claude reply falls through to Sol instead of failing the pass. The tokens lost when every checker replies unusably are the open quest review finding listed under Waiting in the PR body, so this review does not raise them again.

  ^{｜ View workflow run ｜ Using claude-opus-5-5 ｜ 𝕏}

[kixelated]

Maintainer decision (2026-11-14, /quest-run): round-3 design checkpoint

• ✅ Continue: carry the tokens spent by failed checker tries on the error and record them in split.usage when the check is skipped.
• Rejected: label after sending, as a separate pass with its own queue.
• Rejected: price labeling runs through token-max as records of their own.

(Written by Claude Opus 5.5)

[pullfrog]

✅ No new issues found. Tokens spent on unusable replies are now counted even when no model answers, which settles the last open accounting gap.

Reviewed changes

Since the review at f41c370, there is a merge from main and one fix commit. The merge only brings in main's breakage work and its integration in send.ts and usage.ts.

• UnansweredError carries spent: when no model answers usably, ask now throws UnansweredError, a subclass of ReviewInputError, with every turn that replied. When every failure is a usage limit, ask still throws the earliest ReviewLimitError. That loses nothing, because a model at its limit never reaches spent.push.
• labelSession records failed tries: a check no model could give adds its spentBy(error) usage, and the part keeps the splitter's labels. When no splitter could label a part, the code writes labeled with only the added usage before rethrowing. The digest and labels are unchanged, so the next stop's readLabeled keeps that cost and labels the part again.
• Tests: the new ask test checks that both unusable turns appear on the error. The new split test covers an unusable checker and an unusable splitter, and checks that the splitter's cost is still there on the next run.

  ^{｜ View workflow run ｜ Using claude-opus-5-5 ｜ 𝕏}