Skip to content

fix: run the GPU compute benchmark on demand and release its buffer (#11) - #19

Merged
GeoBradDev merged 22 commits into
mainfrom
issue-11-on-demand-gpu-benchmark
Nov 8, 2026
Merged

GeoBradDev merged 22 commits into
mainfrom
issue-11-on-demand-gpu-benchmark

Conversation

@GeoBradDev

Copy link
Copy Markdown
Owner

Makes the GPU compute benchmark on demand. Nothing runs or allocates on mount any more. A RUN button next to COMPUTE in the GPU info panel runs 5 sequential dispatches over the 64 MB buffer, reports the median as ~X.X GB/s est., then destroys the buffer and the device.

  • No overlap: each dispatch waits for onSubmittedWorkDone() before the next. There is no setInterval, and a second activation during a run does nothing (ref guard; the button gets aria-disabled="true" rather than disabled, so focus stays).
  • Hidden tab: a run does not start while document.visibilityState === 'hidden'. Hiding the tab mid-run, including during adapter or device setup, stops it before the next dispatch or before the 64 MB allocation, and shows Stopped. A run whose 5 dispatches already finished keeps its result.
  • Lifetime: buffer and device are destroyed after every run, whether it completed, stopped, or failed, and on unmount.
  • Errors: out-of-memory and validation errors (collected with error scopes) and device loss show Failed instead of a bogus number. A zero median from a coarse timer shows Too fast instead of ~Infinity.
  • Accessibility: native button, accessible name "Run compute benchmark", 2px --accent focus ring, 33.5x24 box, COMPUTE value is an aria-live="polite" region. The panel keeps pointer-events: none; the button opts back in.
  • No WebGPU: the button renders only when the detected tier is webgpu, so with no adapter COMPUTE shows -- with no button, as before.
  • Narrow screens: the button lives inside the secondary COMPUTE item, so at 1024px and below the panel still shows only RENDER, PTS, FRAME (Layout breaks on mobile: header text overlaps and panels overflow at 375px #5 layout unchanged).

Not done, on purpose:

  • Shared GPUDevice with useGPUHeatmap: the heatmap is one-shot and already destroys its device in finally, and the benchmark device now lives only for a user-started run. A shared long-lived device would hold GPU resources longer than either hook does now.
  • timestamp-query: needs an optional feature, a query set, a resolve buffer and a map-read round trip. CPU timing is kept and labelled est..

Plan

docs/plans/2026-11-07-issue-11-on-demand-gpu-benchmark.md, committed at 1dd2147 before any implementation. Revisions during the work, each in the commit that departed:

  • AC1 narrowed to the benchmark hook: the existing useGPUCapabilities probe and the one-shot heatmap already call requestAdapter on mount.
  • AC10: the panel also requires the webgpu tier, because headless Chromium exposes navigator.gpu with no adapter and the hook may not probe on mount.
  • AC4, AC6, AC8, AC10 and Proof extended for the review fixes below (Too fast, stop during setup, async errors and device loss).
  • Frozen test change: the unused setVisibility() and hide() helpers were deleted from src/hooks/useGPUCompute.tooFast.test.js (dead code that broke no-unused-vars; no assertion changed). Approved by the orchestrator under the user's delegation; recorded in the plan.

Test evidence

Gate on 114796c, run verbatim from the worktree root, all exit 0:

  • npm ci: found 0 vulnerabilities
  • npm run lint: eslint ., no output
  • npm test: Test Files 24 passed (24), Tests 211 passed (211)
  • npm run build: built in 1.00s (existing chunk-size warning only)
  • npm audit --omit=dev --audit-level=high: found 0 vulnerabilities

Unit tests with a mocked navigator.gpu cover: no work on mount; 5 strictly sequential dispatches; buffer and device destroyed after complete, stopped and failed runs; hidden before a run, during setup, mid-run, and during the last dispatch; overlap guard; unmount mid-run and during setup; null adapter; error scopes; device loss during a dispatch, together with the last work-done, and during the scope pops. Frozen regression tests: src/components/GPUInfoPanel.benchmarkTier.test.jsx, src/hooks/useGPUCompute.{setupStop,tooFast,asyncErrors,hiddenLast,lostWithWork,lostAfterLoop}.test.js.

Live drive (built app, vite preview, Playwright + Chromium 1254; headless Chromium has navigator.gpu but no adapter, so the WebGPU path used an injected navigator.gpu mock):

  • No WebGPU: RENDER WebGL Fallback, no button, COMPUTE -- (also with navigator.gpu undefined).
  • Keyboard: Tab reaches RUN, :focus-visible outline 2px solid rgb(0, 255, 136), offset 2px; Enter and Space start a run; Running... with aria-disabled="true" and focus kept; result ~0.4 GB/s est. on a 300 ms mock dispatch; 5 submits, 2 error scopes pushed and popped, 1 buffer and 1 device destroyed. Mouse click: same.
  • Edge paths: error scope error shows Failed; device lost (unknown) mid-run, with the 5th work-done, and during the scope pops each show Failed; hidden during requestDevice shows Stopped with no buffer created; hidden after dispatch 2 shows Stopped after 2 submits; hidden during dispatch 5 keeps the result; double click plus Enter plus Space during a run gives 1 adapter request and 5 submits.
  • Layout, with a feature popup open at each size:
Viewport Header overlaps In viewport (header / layer panel / GPU panel / popup) RUN button
375x812 none yes / yes / hidden while popup open / yes; scrollWidth 375 not rendered
768x1024 none yes / yes / yes / yes; popup and GPU panel do not overlap not rendered; panel shows RENDER, PTS, FRAME
1440x900 none yes / yes / yes / yes 33.5x24; GPU panel y 822 to 860, clear of layer panel, popup and attribution

Independent verifier (fresh context, its own mock and scripts) repeated the run, hidden-tab, failure, overlap, no-WebGPU and layout checks plus the layer panel and popup flows: no findings, no console errors.

Real Chrome (user's browser): navigator.gpu is present but requestAdapter() returns null on this machine, so the app shows WebGL Fallback, no RUN button and COMPUTE --. No real throughput number could be measured.

Review findings

First round: three Claude finders (bugs and compliance; security and resource safety; outside the diff) and two Fable finders (bugs and compliance; edge cases), then a Claude confirmation pass. Fixed (source models in brackets):

  • Setup ran after a hide or unmount: device, 64 MB buffer and pipeline were created before the first stop check; run() after unmount did a full run. [claude x2, fable x2]
  • A zero median showed ~Infinity GB/s est.; now Too fast. [claude, fable x2]
  • Async OOM and validation errors and device loss reported a bogus GB/s; now error scopes plus a device.lost race, shown as Failed. [claude, fable]
  • A hide during the last dispatch discarded a complete result. [claude; fable notes]
  • README intro still said "live GPU throughput readout". [claude]
  • An unmount test's console.error check was vacuous under React 19; removed. [claude, fable]

Delta rounds (Claude): device loss settling together with the last work-done still showed GB/s (fixed); loss during the error scope pops still showed GB/s, and the hide-plus-loss case was claimed but untested (fixed, test added). Final delta round: no findings.

Dismissed: sequencing test called weak (the exact-count waitFor already catches an overlapping submit); button border at about 2.9:1 (WCAG 1.4.11 is met by the 13:1 RUN text that identifies the control); layout evidence not in the branch (it is above).

Left as nits: focus drops to body if a run finds no adapter after the tier said WebGPU (rare: the button is then correctly removed); one test sleeps 1.1 s of real time; mock helpers are duplicated across the new test files; one frozen test asserts not.toBe('Failed') where a regex would be stronger.

Docs reconciled

  • README.md: intro, GPU info panel bullet, and Known limitations now describe the on-demand benchmark, its result states, and that RUN is hidden at 1024px and below.
  • CLAUDE.md: no change needed; "a throughput readout" is still accurate.
  • Older plans under docs/: historical, no change.
  • ADR: none.

Closes #11

🤖 Generated with Claude Code

https://claude.ai/code/session_01AELE7jB1i3uqqrHu3SjJjf

GeoBradDev and others added 22 commits November 7, 2026 19:41
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AELE7jB1i3uqqrHu3SjJjf
useGPUCompute no longer allocates or dispatches on mount. run() requests a
device, performs 5 sequential dispatches, reports the median as an
estimate, and destroys the buffer and device afterwards. Runs do not
overlap, stop when the tab is hidden, and clean up on unmount.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AELE7jB1i3uqqrHu3SjJjf
…#11)

The COMPUTE item gets a native button (accessible name "Run compute
benchmark") that is aria-disabled while a run is in progress, and the
COMPUTE value becomes a polite live region. MapView wires it to
useGPUCompute.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AELE7jB1i3uqqrHu3SjJjf
…trast table (#11)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AELE7jB1i3uqqrHu3SjJjf
Browsers can expose navigator.gpu while requestAdapter() returns null.
The benchmark hook may not probe an adapter on mount, so the panel uses
the tier from useGPUCapabilities. The plan's AC1 and AC10 are updated to
match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AELE7jB1i3uqqrHu3SjJjf
… device loss in the GPU mock (#11)

The console.error spy cannot detect a state update after unmount in
React 19, so the unmount test no longer claims to. The mock device gains
pushErrorScope, popErrorScope (resolving null), and a never-settling lost
promise.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AELE7jB1i3uqqrHu3SjJjf
#11)

A hidden tab or unmount while requestAdapter() or requestDevice() is
pending now ends the run before the 64 MB buffer is allocated, and a
device that arrives late is destroyed. run() after unmount does nothing.
The plan's AC6, AC8, and Proof are updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AELE7jB1i3uqqrHu3SjJjf
…#11)

The plan's AC4 and Proof are updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AELE7jB1i3uqqrHu3SjJjf
The run pushes out-of-memory and validation error scopes before setup and
pops them after the loop, and races each onSubmittedWorkDone() against
device.lost. A scope error or a loss not caused by our own destroy shows
Failed. The plan's AC10 and Proof are updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AELE7jB1i3uqqrHu3SjJjf
The outcome now depends only on whether all 5 timings were collected. The
plan's AC6 and Proof are updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AELE7jB1i3uqqrHu3SjJjf
Approved by the orchestrator under the user's delegation: the unused
setVisibility() and hide() helpers, copied from the shared mock block,
broke lint. No assertion changed. Recorded in the plan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AELE7jB1i3uqqrHu3SjJjf
device.lost can settle in the same tick as the pending work-done, so the
race may record a cut-short timing. After the loop, any recorded loss
other than our own destroy now shows Failed. The plan's Proof is updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AELE7jB1i3uqqrHu3SjJjf
Also covers a hide plus a device loss mid-run showing Failed, which the
plan's Proof already claimed but no test checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AELE7jB1i3uqqrHu3SjJjf
A loss that settles while the scopes are popped (they resolve null on a
lost device) now shows Failed. The in-loop assignment is dropped because
the post-pop check covers it. The plan's AC10 and Proof are updated,
including the hide plus loss case that is now tested.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AELE7jB1i3uqqrHu3SjJjf
@GeoBradDev
GeoBradDev merged commit 67ca6dc into main Nov 8, 2026
1 check passed
@GeoBradDev
GeoBradDev deleted the issue-11-on-demand-gpu-benchmark branch November 8, 2026 02:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GPU compute benchmark runs a 64 MB dispatch every second indefinitely, including in hidden tabs

1 participant