Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions docs/plans/2026-11-07-issue-15-gpu-heatmap-misregistered.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Plan: GPU heatmap is misregistered: equirectangular grid drawn as a Mercator-stretched bitmap (issue #15)

Risk tier: High. Starts Low, raised once because the visible effect (heat lining up with airports on the rendered map) is a GPU and map-rendering result no unit test in jsdom can exercise, and once because it alters existing behavior (the shader's binning). Small diff, but rounded up per the rules.

Domain skills:
- superpowers:test-driven-development: invoke (every change starts with a failing test).
- context7 / deck.gl docs: used in analysis (see Approach); no further invocation needed.
- vercel-react-best-practices: not applicable because no React component or hook structure changes; only a shader string and layer props change.
- dataviz: not applicable because the color ramp and opacity do not change; only where pixels land.
- frontend-design, a11y skills: not applicable because no UI, markup, or styles change.

## Approach

Bin in Web Mercator inside the shader and keep the BitmapLayer's current Mercator bounds `[-180, -85.051129, 180, 85.051129]` with the default image coordinate system.

Rejected: `_imageCoordinateSystem: COORDINATE_SYSTEM.LNGLAT` with bounds `[-180, -90, 180, 90]`. In deck.gl 9.4.1 (`node_modules/@deck.gl/layers/dist/bitmap-layer/bitmap-layer.js` line 177 onward, `bitmap-layer-fragment.js` line 52) it is an underscore (experimental) prop, it reprojects per fragment while clamping latitude to +/-89.9, and the quad corners at +/-90 project to infinite Mercator Y. Whether that renders correctly in the non-interleaved MapboxOverlay is uncertain, and nothing in the repo can pin it in a unit test. Mercator binning is exact, deterministic, and testable in plain JS.

## Acceptance criteria

1. Heatmap hot spots line up with the airport points at all latitudes up to +/-80 (verify visually at about 0, 40, and 65 degrees N).
2. The latitude mapping the shader uses and the bounds and projection of the BitmapLayer agree, and a unit test covers that.

Checkable form:
- 1a. For any latitude in [-80, 80], the cell row the binning maps it to, when drawn by a BitmapLayer stretched linearly in Mercator Y over the layer's bounds, covers a latitude band that contains the input latitude (unit test over a sweep, plus named reference points 0, 40, 60, 65, -33.9).
- 1b. In a real browser with a WebGPU adapter (Chromium, `--enable-unsafe-webgpu --use-webgpu-adapter=swiftshader`), screenshots with HEATMAP and AIRPORTS on at about 0N (West Africa), 40N (US mid-Atlantic), and 65N (Scandinavia or Alaska) show hot spots on the airport dots, not shifted north.
- 2a. One JS module owns the Mercator latitude limit, the grid dimensions used for rows, the lat-to-row formula, and the BitmapLayer bounds. `buildLayers` uses its bounds and sets no `_imageCoordinateSystem`. The accumulate shader's WGSL binning code is generated from that module's constants (no second hand-typed copy of 85.051129). The WGSL bodies do restate the row and column formulas in WGSL syntax, so a test executes the generated WGSL (2b) to prove they match the JS helpers.
- 2b. A unit test asserts the BitmapLayer's bounds equal the helper's bounds, that the numeric constants embedded in the generated WGSL equal the JS constants, and that the old linear `(90.0 - lat) / 180.0` mapping is gone. A second test translates the generated WGSL `lonToCol` and `latToRow` mechanically into JS (failing on any identifier, character, parameter type or return type it does not know; numeric literals are read as floats, which holds because the generator emits every literal with a decimal point) and asserts they return the same values as the JS helpers at the reference points, at and beyond +/-85.051129 and +/-90, and over sweeps of lat -85..85 and lon -180..180.
- 2c. Latitudes beyond +/-85.051129 (and +/-90) clamp to the first or last row instead of producing NaN or an out-of-range row.

## Files that change

- `src/layers/heatmapProjection.js` (new): exports `MERCATOR_MAX_LAT = 85.051129`, `HEATMAP_GRID_W = 1024`, `HEATMAP_GRID_H = 512`, `HEATMAP_BOUNDS` (`[-180, -MERCATOR_MAX_LAT, 180, MERCATOR_MAX_LAT]`), `mercatorY(lat)` (`ln(tan(pi/4 + lat*pi/360))` with lat clamped to +/-MERCATOR_MAX_LAT), `latToRowFraction(lat)` (`(1 - mercatorY(lat) / mercatorY(MERCATOR_MAX_LAT)) / 2`, 0 at the north edge, 1 at the south edge), `latToRow(lat, gridH = HEATMAP_GRID_H)` (floor of fraction times gridH, clamped to [0, gridH-1]), `lonToCol(lon, gridW = HEATMAP_GRID_W)` (same as the current shader), and `binningWGSL()` (or a constant string) returning WGSL functions `lonToCol(lon: f32) -> u32` and `latToRow(lat: f32) -> u32` that use the same formula, with the limit, its Mercator Y, and the grid sizes interpolated from the JS values as f32 literals (always with a decimal point).
- `src/layers/heatmapProjection.test.js` (new): reference points and the agreement sweep (1a), clamping (2c), WGSL constants (2b).
- `src/layers/heatmapProjection.wgsl.test.js` (new, added after review): executes the generated WGSL binning functions via a strict WGSL-to-JS translator (each rejection test pins the error message of the path it names) and compares them with the JS helpers: exactly in float64 (2b), and under f32 emulation (Math.fround on every literal, parameter, operator and builtin result) asserting rows and columns stay in the grid, extremes land on the first or last row and column, and sweeps stay within one cell of the JS helpers (2c).
- `src/hooks/useGPUHeatmap.js`: import the grid sizes and WGSL from the helper, drop the local `GRID_W`/`GRID_H` constants, call the generated `lonToCol`/`latToRow` in `ACCUMULATE_SHADER`, and export `ACCUMULATE_SHADER` so a test can inspect it. Nothing else in the hook changes.
- `src/hooks/useGPUHeatmap.test.js`: add a test that `ACCUMULATE_SHADER` contains the generated binning code and no `(90.0 - lat) / 180.0`.
- `src/layers/index.js`: `bounds: HEATMAP_BOUNDS`.
- `src/layers/index.test.js`: assert heatmap layer bounds equal `HEATMAP_BOUNDS` and `_imageCoordinateSystem` is undefined.
- `README.md`: only if a line becomes false (Phase 8 decides).

## Order of work

1. Red: write `heatmapProjection.test.js` and the new assertions in `index.test.js` and `useGPUHeatmap.test.js`. Run `npm test`; record the failures.
2. Green: add `heatmapProjection.js`, wire it into `index.js` and `useGPUHeatmap.js`.
3. Full gate (`npm ci`, `npm run lint`, `npm test`, `npm run build`, `npm audit --omit=dev --audit-level=high`).
4. Live drive (below), on `npm run dev` or `npm run preview`.

## Risks

- f32 precision in WGSL: `log(tan(...))` near 85 degrees is fine in f32; the clamp to +/-MERCATOR_MAX_LAT keeps `tan` finite. The latitude is clamped to +/-MERCATOR_MAX_LAT first, so the fraction is always finite and in [0, 1]; the row is the floor of fraction times gridH, then clamped to [0, gridH-1] (this maps the south limit, fraction 1, to the last row).
- Floating literal formatting: a JS integer like 1024 interpolated as `1024` is an abstract int in WGSL; emit `1024.0` style f32 literals where floats are expected to avoid type errors. The live drive proves the shader compiles.
- Cell shape: a 1024x512 grid over the square Mercator world gives cells twice as tall as wide. Kept as is to stay in scope; alignment is what the issue asks for.
- Do not touch the hook's run-once behavior, buffer lifetimes, or the COLORIZE shader. Do not touch other prior work (a11y, layout, contrast tests, benchmark, RENDER label).

What I chose not to do: change grid resolution, change the color ramp, add zoom-scaled radius (#9), or loading and error states (#10).

## Proof

- Criterion 1a, 2c: `src/layers/heatmapProjection.test.js`.
- Criterion 2a, 2b: `src/layers/index.test.js` and `src/hooks/useGPUHeatmap.test.js` plus `heatmapProjection.test.js` WGSL-constant tests, and `heatmapProjection.wgsl.test.js`, which executes the generated WGSL row and column formulas and compares them with the JS helpers (shown to fail when the WGSL `/ 2.0` becomes `* 2.0`, when `1.0 -` is dropped, or when `+ 180.0` is dropped).
- Criterion 2c on the GPU path: the f32-emulated tests in `heatmapProjection.wgsl.test.js` (in f32 the fraction at -85.051129 is exactly 1 and at +85.051129 slightly negative, so the row clamp is what keeps rows in [0, 511]; shown to fail when the upper clamp becomes `512.0` or `600.0` or the lower clamp becomes `-5.0`).
- Criterion 1b: live drive in Playwright Chromium with a SwiftShader WebGPU adapter: HEATMAP and AIRPORTS on, screenshots at about 0N (e.g. lon 0, lat 7, zoom 4), 40N (lon -76, lat 39, zoom 5), and 65N (lon 18, lat 63, zoom 4 or Alaska lon -150, lat 63), plus a numeric check in the page: for a few airports, the heat pixel's screen position (from the rendered bitmap) is within one cell of `map.project([lon, lat])`. Before-and-after screenshots at 60N or 65N show the shift is gone.
14 changes: 6 additions & 8 deletions src/hooks/useGPUHeatmap.js
Original file line number Diff line number Diff line change
@@ -1,19 +1,18 @@
import { useState, useEffect, useRef } from 'react'
import { HEATMAP_GRID_W as GRID_W, HEATMAP_GRID_H as GRID_H, binningWGSL } from '../layers/heatmapProjection.js'

const GRID_W = 1024
const GRID_H = 512
const GRID_SIZE = GRID_W * GRID_H
const WORKGROUP_SIZE = 256

// ---- WGSL shaders --------------------------------------------------------

const ACCUMULATE_SHADER = `
export const ACCUMULATE_SHADER = `
struct Params { count: u32 }

@group(0) @binding(0) var<storage, read> points: array<vec2<f32>>;
@group(0) @binding(1) var<uniform> params: Params;
@group(0) @binding(2) var<storage, read_write> grid: array<atomic<u32>>;

${binningWGSL()}
@compute @workgroup_size(${WORKGROUP_SIZE})
fn main(@builtin(global_invocation_id) gid: vec3<u32>) {
let i = gid.x;
Expand All @@ -22,10 +21,9 @@ fn main(@builtin(global_invocation_id) gid: vec3<u32>) {
let lon = points[i].x;
let lat = points[i].y;

// Map lon [-180, 180] -> col [0, GRID_W-1]
let col = u32(clamp((lon + 180.0) / 360.0 * f32(${GRID_W}), 0.0, f32(${GRID_W} - 1u)));
// Map lat [-90, 90] -> row [0, GRID_H-1] (north at row 0)
let row = u32(clamp((90.0 - lat) / 180.0 * f32(${GRID_H}), 0.0, f32(${GRID_H} - 1u)));
// Bin in Web Mercator to match the BitmapLayer bounds (see heatmapProjection.js)
let col = lonToCol(lon);
let row = latToRow(lat);

atomicAdd(&grid[row * ${GRID_W}u + col], 1u);
}
Expand Down
19 changes: 17 additions & 2 deletions src/hooks/useGPUHeatmap.test.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { describe, it, expect } from 'vitest'
import { renderHook } from '@testing-library/react'
import { useGPUHeatmap, buildPointsData } from './useGPUHeatmap.js'
import { useGPUHeatmap, buildPointsData, ACCUMULATE_SHADER } from './useGPUHeatmap.js'
import { binningWGSL } from '../layers/heatmapProjection.js'

describe('buildPointsData', () => {
it('returns a Float32Array of length airportCount * 2', () => {
Expand Down Expand Up @@ -44,4 +45,18 @@ describe('useGPUHeatmap', () => {
it('does not throw when airports array is empty', () => {
expect(() => renderHook(() => useGPUHeatmap([]))).not.toThrow()
})
})
})
describe('ACCUMULATE_SHADER binning', () => {
it('includes the generated Mercator binning functions', () => {
expect(ACCUMULATE_SHADER).toContain(binningWGSL())
})

it('bins each point with the generated lonToCol and latToRow', () => {
expect(ACCUMULATE_SHADER).toMatch(/lonToCol\(\s*lon\s*\)/)
expect(ACCUMULATE_SHADER).toMatch(/latToRow\(\s*lat\s*\)/)
})

it('no longer uses the linear equirectangular latitude mapping', () => {
expect(ACCUMULATE_SHADER).not.toContain('(90.0 - lat) / 180.0')
})
})
84 changes: 84 additions & 0 deletions src/layers/heatmapProjection.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
// Single source of truth for how the GPU heatmap grid maps to the map.
//
// The heatmap image is drawn by a deck.gl BitmapLayer over HEATMAP_BOUNDS with
// the default image coordinate system, which stretches the image linearly in
// Web Mercator Y. The accumulate shader therefore bins latitude in Mercator Y
// too, using WGSL generated from the constants below, so a point's cell is
// drawn where the point is.

export const MERCATOR_MAX_LAT = 85.051129
export const HEATMAP_GRID_W = 1024
export const HEATMAP_GRID_H = 512
export const HEATMAP_BOUNDS = [-180, -MERCATOR_MAX_LAT, 180, MERCATOR_MAX_LAT]

const QUARTER_PI = Math.PI / 4
const HALF_DEG_TO_RAD = Math.PI / 360

/**
* Web Mercator Y (in radians of the unit sphere) for a latitude in degrees,
* with latitude clamped to +/-MERCATOR_MAX_LAT so the result is always finite.
* @param {number} lat
* @returns {number}
*/
export function mercatorY(lat) {
const clamped = Math.min(Math.max(lat, -MERCATOR_MAX_LAT), MERCATOR_MAX_LAT)
return Math.log(Math.tan(QUARTER_PI + clamped * HALF_DEG_TO_RAD))
}

const MERCATOR_MAX_Y = mercatorY(MERCATOR_MAX_LAT)

/**
* Fraction of the image height from the north edge (0) to the south edge (1).
* @param {number} lat
* @returns {number}
*/
export function latToRowFraction(lat) {
return (1 - mercatorY(lat) / MERCATOR_MAX_Y) / 2
}

/**
* @param {number} lat
* @param {number} [gridH]
* @returns {number} row in [0, gridH - 1], north at row 0
*/
export function latToRow(lat, gridH = HEATMAP_GRID_H) {
const row = Math.floor(latToRowFraction(lat) * gridH)
return Math.min(Math.max(row, 0), gridH - 1)
}

/**
* @param {number} lon
* @param {number} [gridW]
* @returns {number} column in [0, gridW - 1]
*/
export function lonToCol(lon, gridW = HEATMAP_GRID_W) {
const col = Math.floor(((lon + 180) / 360) * gridW)
return Math.min(Math.max(col, 0), gridW - 1)
}

/** Format a JS number as a WGSL f32 literal (always with a decimal point). */
function f32(value) {
return Number.isInteger(value) ? `${value}.0` : String(value)
}

/**
* WGSL functions `lonToCol(lon: f32) -> u32` and `latToRow(lat: f32) -> u32`
* matching the JS helpers above, with every constant interpolated from JS.
* @returns {string}
*/
export function binningWGSL() {
return `
fn lonToCol(lon: f32) -> u32 {
let c = floor((lon + 180.0) / 360.0 * ${f32(HEATMAP_GRID_W)});
return u32(clamp(c, 0.0, ${f32(HEATMAP_GRID_W - 1)}));
}

fn latToRow(lat: f32) -> u32 {
let clamped = clamp(lat, ${f32(-MERCATOR_MAX_LAT)}, ${f32(MERCATOR_MAX_LAT)});
let y = log(tan(${f32(QUARTER_PI)} + clamped * ${f32(HALF_DEG_TO_RAD)}));
let fraction = (1.0 - y / ${f32(MERCATOR_MAX_Y)}) / 2.0;
let r = floor(fraction * ${f32(HEATMAP_GRID_H)});
return u32(clamp(r, 0.0, ${f32(HEATMAP_GRID_H - 1)}));
}
`
}
167 changes: 167 additions & 0 deletions src/layers/heatmapProjection.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
import { describe, it, expect } from 'vitest'
import {
MERCATOR_MAX_LAT,
HEATMAP_GRID_W,
HEATMAP_GRID_H,
HEATMAP_BOUNDS,
mercatorY,
latToRowFraction,
latToRow,
lonToCol,
binningWGSL,
} from './heatmapProjection.js'

// Independent reference math, written here so the test does not trust the module.
const DEG = Math.PI / 180
const refMercY = (lat) => Math.log(Math.tan(Math.PI / 4 + (lat * DEG) / 2))
const refInvMercY = (y) => (2 * Math.atan(Math.exp(y)) - Math.PI / 2) / DEG

/**
* The latitude band a BitmapLayer draws image row `row` over, when the image
* is stretched linearly in Mercator Y across [south, north] (deck.gl default
* image coordinate system for a non-geospatial-reprojected bitmap).
*/
function rowBand(row, bounds, gridH) {
const yTop = refMercY(bounds[3])
const yBottom = refMercY(bounds[1])
const step = (yTop - yBottom) / gridH
const north = refInvMercY(yTop - row * step)
const south = refInvMercY(yTop - (row + 1) * step)
return { north, south }
}

describe('heatmapProjection constants', () => {
it('uses the Web Mercator latitude limit', () => {
expect(MERCATOR_MAX_LAT).toBe(85.051129)
})

it('keeps the existing grid size', () => {
expect(HEATMAP_GRID_W).toBe(1024)
expect(HEATMAP_GRID_H).toBe(512)
})

it('bounds span the full Mercator world', () => {
expect(HEATMAP_BOUNDS).toEqual([-180, -MERCATOR_MAX_LAT, 180, MERCATOR_MAX_LAT])
})

it('mercatorY matches the reference formula', () => {
for (const lat of [-80, -33.9, 0, 40, 65, 80]) {
expect(mercatorY(lat)).toBeCloseTo(refMercY(lat), 10)
}
})

it('the Mercator limit is the latitude whose Mercator Y is pi', () => {
expect(mercatorY(MERCATOR_MAX_LAT)).toBeCloseTo(Math.PI, 5)
})
})

describe('latToRow agrees with a Mercator-stretched bitmap (criterion 1a)', () => {
it('equator lands on the middle rows', () => {
expect(latToRowFraction(0)).toBeCloseTo(0.5, 10)
expect(latToRow(0)).toBe(HEATMAP_GRID_H / 2)
})

it('reference latitudes fall inside the band their row is drawn over', () => {
for (const lat of [0, 40, 60, 65, -33.9]) {
const row = latToRow(lat)
const { north, south } = rowBand(row, HEATMAP_BOUNDS, HEATMAP_GRID_H)
expect(lat).toBeLessThanOrEqual(north + 1e-9)
expect(lat).toBeGreaterThanOrEqual(south - 1e-9)
}
})

it('65N maps to a row far from what the old linear mapping gave', () => {
// Old: floor((90 - 65) / 180 * 512) = 71. Under Mercator it is much nearer the top.
const oldRow = Math.floor(((90 - 65) / 180) * HEATMAP_GRID_H)
expect(oldRow).toBe(71)
expect(latToRow(65)).not.toBe(oldRow)
const { north, south } = rowBand(oldRow, HEATMAP_BOUNDS, HEATMAP_GRID_H)
// The old row is drawn north of 65N, which is the reported shift.
expect(south).toBeGreaterThan(65)
expect(north).toBeGreaterThan(65)
})

it('every latitude in [-80, 80] lands in the band its row is drawn over', () => {
for (let lat = -80; lat <= 80; lat += 0.05) {
const row = latToRow(lat)
const { north, south } = rowBand(row, HEATMAP_BOUNDS, HEATMAP_GRID_H)
expect(lat).toBeLessThanOrEqual(north + 1e-9)
expect(lat).toBeGreaterThanOrEqual(south - 1e-9)
}
})

it('rows increase monotonically southward', () => {
let prev = -1
for (let lat = 85; lat >= -85; lat -= 0.5) {
const row = latToRow(lat)
expect(row).toBeGreaterThanOrEqual(prev)
prev = row
}
})
})

describe('clamping (criterion 2c)', () => {
it('latitudes at or beyond the north limit clamp to row 0', () => {
for (const lat of [MERCATOR_MAX_LAT, 86, 89.999, 90, 120]) {
expect(latToRow(lat)).toBe(0)
}
})

it('latitudes at or beyond the south limit clamp to the last row', () => {
for (const lat of [-MERCATOR_MAX_LAT, -86, -90, -120]) {
expect(latToRow(lat)).toBe(HEATMAP_GRID_H - 1)
}
})

it('never returns NaN for extreme latitudes', () => {
for (const lat of [90, -90]) {
expect(Number.isFinite(mercatorY(lat))).toBe(true)
expect(Number.isFinite(latToRowFraction(lat))).toBe(true)
}
})

it('lonToCol clamps to [0, gridW - 1]', () => {
expect(lonToCol(-180)).toBe(0)
expect(lonToCol(180)).toBe(HEATMAP_GRID_W - 1)
expect(lonToCol(0)).toBe(HEATMAP_GRID_W / 2)
expect(lonToCol(-500)).toBe(0)
expect(lonToCol(500)).toBe(HEATMAP_GRID_W - 1)
})

it('latToRow and lonToCol honor custom grid sizes', () => {
expect(latToRow(0, 8)).toBe(4)
expect(latToRow(90, 8)).toBe(0)
expect(latToRow(-90, 8)).toBe(7)
expect(lonToCol(0, 8)).toBe(4)
})
})

describe('binningWGSL (criterion 2b)', () => {
const wgsl = binningWGSL()
const floats = (wgsl.match(/-?\d+\.\d+/g) ?? []).map(Number)

it('declares lonToCol and latToRow WGSL functions', () => {
expect(wgsl).toMatch(/fn\s+lonToCol\s*\(\s*lon\s*:\s*f32\s*\)\s*->\s*u32/)
expect(wgsl).toMatch(/fn\s+latToRow\s*\(\s*lat\s*:\s*f32\s*\)\s*->\s*u32/)
})

it('embeds the JS latitude limit', () => {
expect(floats).toContain(MERCATOR_MAX_LAT)
})

it('embeds the Mercator Y of the limit computed in JS', () => {
const target = mercatorY(MERCATOR_MAX_LAT)
expect(floats.some((v) => Math.abs(v - target) < 1e-9)).toBe(true)
})

it('embeds the grid sizes as f32 literals with a decimal point', () => {
expect(wgsl).toContain(`${HEATMAP_GRID_W}.0`)
expect(wgsl).toContain(`${HEATMAP_GRID_H}.0`)
})

it('uses a Mercator log-tan formula, not the old linear one', () => {
expect(wgsl).toMatch(/log\s*\(\s*tan\s*\(/)
expect(wgsl).not.toContain('(90.0 - lat) / 180.0')
expect(wgsl).not.toMatch(/\b90\.0\b/)
})
})
Loading
Loading