[sswiftdrtv]

Why

#1234 decoded a batch's prompts together on the GPU to make full_report faster. It was merged without a GPU run, with a stated fallback: verify on production and revert if it didn't help. In production it doubled the time and the billed seconds.

Same song: a public-domain 1915 Edison recording, 3:59. Each run followed a fresh Modal deploy.

full_report	after #1233 (one at a time)	after #1234 (grouped)
Wall time	181 s	367 s
Billed seconds	179	365
Seconds charge (approx.)	$0.21	$0.43

Moment descriptions barely moved (80 s → 75 s). The greedy presets' answers drifted by a few tokens, as #1234 expected, so the problem is speed and cost, not answer quality.

Callers can't fall back to one-at-a-time decoding: group_size is a Modal field the API doesn't send. So the revert is the only fallback.

What changes

This reverts the merge commit 026071c. generate_batch goes back to answering prompts one at a time: the code verified in production after #1233. Only modal/ changes, and its three files match main before #1234.

Follow-up

Find out where the time goes before trying grouped decoding again. Candidates:

• OOM split-and-retry on a long song, where failed attempts still count toward elapsed time.
• A whole group waiting on its longest answer (lyrics).

#1234's bench entrypoint can measure this, but needs access to the shared Modal workspace.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
api	Ready	Preview	Nov 10, 2026 1:25am UTC

[coderabbitai]

Important

Review skipped

Review was skipped due to path filters

⛔ Files ignored due to path filters: 3

• modal/README.md is excluded by none and included by none
• modal/serve_music_flamingo.py is excluded by none and included by none
• modal/test_serve_music_flamingo.py is excluded by none and included by none

CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including **/dist/** will override the default block on the dist directory, by removing the pattern from both the lists.

Run configuration

• Source mode: Auto
• Repository: recoupable/api
• Branch: main
• File: .coderabbit.yaml
• Commit: 026071c
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Advanced
• Run ID: 3298159b-0667-4e9a-ba22-b8ab5169a3b9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[chatgpt-codex-connector]

Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-10T01:26:30.221508Z	ba8b808	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.