Skip to content

Image QC metrics per plane and channel, computed in the histogram pass - #365

Merged
guijacquemet merged 9 commits into
mainfrom
claude/image-qc-metrics
Nov 22, 2026
Merged

guijacquemet merged 9 commits into
mainfrom
claude/image-qc-metrics

Conversation

@guijacquemet

Copy link
Copy Markdown
Contributor

Summary

Per-plane, per-channel image QC metrics (design metrics 1-8), measured in the histogram pass from the blocks the histogram already reads (no extra file read), written to a new run-folder table image_qc.csv and named in the manifest (image_qc: file, SHA-256, qc_version, settings).

  • image_qc.py (pure): PlaneQc accumulates combinable sums block by block; finish(histogram) adds the histogram metrics; ImageQc holds the values (None = undefined, written blank, never zero).
  • image_qc_sums.py (pure): BlockSums, one cache-blocked sweep per block. Every neighbour metric is reduced to shifted dot products (10 offsets) plus thin border strips, so the cost is one pass over memory.
  • image_qc_moments.py (pure): Moments / CoMoments, combined across blocks exactly (pairwise Chan update).
  • image_qc_table.py (io): the image_qc.csv writer and strict reader. image_qc_record.py (pure): the manifest entry, read strictly.
  • run/rows.py: _measure taps the histogram's first pass over each channel plane. planes.py is not touched.
  • Reuse and resume follow the histogram step:
    • reused rows keep their QC rows;
    • staging is now version 2, so <id>.qc.csv is staged with the rows (version 1 records are discarded and those images read again);
    • an earlier table that is missing, changed, of another qc_version or settings, or not matching the histogram table recomputes every row, with the reason in the report.
  • run/generation.py, run/pipeline.py, log_target.py: the table is part of the generation and protected from --log.
  • Docs: contract §19 (new) plus the §9 manifest row and the §9.4 staging and generation rows; user guide §6; CHANGELOG; AGENTS repo map; development.md (benchmark tool).

Not touched: channel_identity.py (PR #364) and notebook_channels_repeat.py. Feeding the shape metrics into channel identity is a later PR.

Design (binding): MorphoFeature claude/restart-doc, docs/restart/designs/image-qc-metrics-2026-11-21.md

Metric definitions (qc_version 1)

m = plane mean. Every ratio metric is divided by m or m², so it is scale-free. Ratio metrics are blank when m <= 0.

Column Definition
otsu_threshold Two-class Otsu on the plane's histogram (each bin at its representative value, robust tails at their mean; exact for 8/16-bit default bins): the value of the highest bin of the lower class
otsu_separation Between-class over total variance at the threshold (0-1). Not in the design, see choices
foreground_fraction Share of finite pixels above the threshold
entropy Shannon entropy (bits) over 64 equal bins from plane min to max
normalized_variance var / m² (see choices)
tenengrad (mean gx² + mean gy²) / m², central differences (I[x+1]-I[x-1])/2
laplacian_variance Variance of the 4-neighbour Laplacian over inner pixels, / m²
correlation_1/4/16 Pearson correlation with the neighbour k px right and k px below, mean of the two directions
local_focus_median, local_focus_p10 Tile variance / m² (plane mean) on a fixed 8 x 8 grid; median and 10th percentile (linear) over 64 tiles; blank when a side is < 16 px
tile_mean_cv Population std / mean of the 64 tile means

qc_status is one of:

  • measured;
  • non_finite_pixels: pixel metrics blank, histogram metrics on the finite pixels;
  • no_finite_pixels: all blank.

For blocked planes (read_in_blocks true), the histogram, moment and tile metrics equal the whole plane's. The neighbour metrics leave out pairs that cross a block edge (tested: within 5% for Tenengrad and the Laplacian, and within 0.05 absolute for the correlations, on blocks down to 17 x 33 px).

Benchmark

tools/benchmark_image_qc.py (committed). Setup:

  • 10 uncompressed OME-TIFFs, each 4 x 2048 x 2048 uint16; plus 1 such image read in blocks (memory limit 16 MiB);
  • mode="full", 1 worker;
  • 10 rounds plus 1 warm-up, ABBA order, each run in a fresh process;
  • baseline: origin/main 68a3952 through PYTHONPATH;
  • histogram step = run.steps.new_rows (read, histogram, QC, staging).

plane_histograms.csv and experiment_design.csv are byte-identical in all 40 runs, and all runs with the metrics wrote the same image_qc.csv (identical_outputs: true). The machine was shared (load average 1.2-2.4 during the runs; not pinned).

Configuration Histogram step, median (min-max) Whole run wall, median
without, whole (40 channel planes) 3.97 s (3.36-4.75) 5.32 s
with, whole 5.36 s (4.27-6.00) 6.66 s
paired ratio, whole 1.32 (1.15-1.60)
without, blocks (4 channel planes) 0.398 s (0.296-0.455) 0.80 s
with, blocks 0.533 s (0.439-0.645) 0.88 s
paired ratio, blocks 1.39 (1.14-2.13)

The median is slightly over the +30% target (+32% whole, +39% blocks; noisy machine). The owner decides whether that is acceptable (see choices).

What was tried, from earlier rounds on the same data:

Kernel Whole histogram step
Plain einsum over the whole plane about +67%
Multithreaded BLAS dot on bands very slow: thread start-up on the loaded machine
Bands of 64 rows with numpy.vecdot over rows of 8192 (final kernel) +32%

The final kernel's own cost is about 19-26 ms per 2048² plane.

PLLS is not included. Measured alone, scipy.fft.rfft2 plus power on one 2048 x 2048 plane takes 29 ms (float32) to 62 ms (float64), before any radial averaging. That is +40-85% of the ~74 ms per channel plane histogram step on its own, so the total cannot stay within +50%. This is a micro-benchmark of the FFT only, not a pipeline run.

Choices for the owner

  1. normalized_variance = var / m², not CellProfiler's var / m. The design names CellProfiler's FocusScore (var / m), but that scales with intensity, which contradicts the design's scale-free principle and its brightness-invariance validation. Alternative: keep var / m and exclude it from the invariance rule.
  2. Local focus divides each tile's variance by the plane mean², not the tile's own mean (CellProfiler). This avoids blow-ups and undefined values on dark or empty tiles. Alternative: per-tile mean, with dark tiles excluded.
  3. Added otsu_separation. Two-class Otsu always splits, so on a truly empty field (unimodal noise) foreground_fraction is about 0.5, not near 0 as the design's validation expects. A near-empty field (noise plus a speck of debris) does give < 1%. otsu_separation (Otsu's own separability) tells the two cases apart (test test_otsu_on_noise_alone_splits_the_noise_with_a_low_separation). Alternative: drop it and document the limitation only.
  4. Otsu and entropy come from the table's histogram bins (robust tails at their mean). They are exact for 8/16-bit default bins; for float data they depend on the 10,000-bin binning (uint16 vs float/65535: foreground_fraction within 0.01, entropy within 0.05).
  5. A float plane with any NaN/inf gets blank pixel metrics (non_finite_pixels), rather than a masked computation.
  6. Missing earlier table means a full recompute once. When an earlier run has no image_qc entry (every existing output folder), the first rerun recomputes every row ("the earlier run has no image QC table"), rather than rereading only for QC.
  7. The table holds plane identity columns plus channel, sorted by image, z, t, channel ID. Population statistics are used (ddof 0); entropy is in bits.
  8. Benchmark result: +32% / +39% against the +30% target. Options: accept it, or drop the Laplacian (its two diagonal offsets) and the lag-2 offsets, which would save about 4 of 10 dot passes.
  9. Determinism. Integer planes give sums that are exact whatever the summation order (below 2**53). Float planes are byte-identical on the same machine; another CPU may change the last digits. Workers 1 vs 2 give identical tables (tested).
  10. RunInputs does not read or check image_qc.csv, and stale-folder refusals ignore it, since the table is information only. If later bricks consume it, RunInputs should check its SHA-256.

Spec impact

None needed. The Spec checkout (MorphoBricks-Spec main 232fb38) does not enumerate the preprocessing run folder's files:

  • ADR 0007's context names run_manifest.json and the histogram table only.
  • Its RUN_FOLDER_STALE lists the files RunInputs checks, which this PR does not extend.
  • The run-folder inventory design lists no file table.

So there is no Spec PR. If the owner wants the table enumerated (for example when channel identity starts using it), a one-line addition to ADR 0007's validated-run-folder context would do.

Tests

  • tests/test_image_qc.py (27 tests):
    • brute-force pair and window sums;
    • every metric against an independent direct implementation;
    • hand-computed known values per formula (normalized variance, Tenengrad, Laplacian, checkerboard correlations, Otsu, separation and entropy, tile metrics, constant plane);
    • monotonic Gaussian blur series;
    • x0.5 and x2 brightness, uint16 vs float32/65535, integer doubling;
    • near-empty field and pure noise;
    • left-right illumination ramp;
    • whole vs 4 block shapes, and full-width bands;
    • coverage check, non-finite, all-NaN and negative mean.
  • tests/test_image_qc_run.py (22 tests):
    • table and manifest entry match the histogram table, and each row equals a direct measurement;
    • blocked run: read_in_blocks, identical histograms;
    • rerun reuses without reading and leaves the bytes unchanged;
    • a changed image replaces only its rows, and the result equals a full run;
    • edited, deleted, unrecorded or other-version table forces a recompute, with the reason;
    • interrupted run resumes staged QC rows, and the result equals the reference;
    • workers 2 equals serial;
    • --log protection;
    • table round trip and strict reader;
    • strict manifest entry.

Local check

python tools/check.py in the worktree with a fresh venv (.[dev,notebook,zarr]), no PYTEST_ADDOPTS, fresh TMPDIR. Ruff check and format pass, and the code rules pass (the baseline is lowered by one line for tests/test_notebook_ui.py). Pytest: 6840 passed, 16 skipped, 0 failed. The skips are environment skips:

  • 11: scikit-image not installed (he-mask extra);
  • 3: tests for case-insensitive file systems (this one is case-sensitive);
  • 1: running with permissions that ignore file modes;
  • 1: editable install code-identity check.

Not verified:

  • CI (Windows and macOS, Python 3.11 with zarr 2.18);
  • real microscope images (synthetic data only);
  • comparison against CellProfiler MeasureImageQuality (design: later).

https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm


Generated by Claude Code

Design metrics 1-8 (MorphoFeature image-qc-metrics-2026-11-21): Otsu
foreground fraction (with threshold and separation), entropy, normalized
variance, Tenengrad, Laplacian variance, correlation at lags 1/4/16,
local focus median/p10 and tile mean CV. They are scale-free and measured
from the blocks the histogram already reads (no extra read), and written to
the new run-folder table image_qc.csv, which the manifest names under the
optional key image_qc. The table follows the histogram step's reuse and
resume rules (staging version 2). PLLS is left out because of its cost.
Includes contract section 19, the user guide, the CHANGELOG and
tools/benchmark_image_qc.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm
Resolves CHANGELOG (both entries kept) and manifest.py (main's docstring notes kept).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm
B1 numpy>=2.0 floor (numpy.vecdot), with the other floors raised to releases that import
with NumPy 2 (scipy 1.13, matplotlib 3.8.4, scikit-image 0.24 for he-mask).
B2 a changed memory limit recomputes rows when planes are read in blocks (run.reuse
_settings_changed) and staged rows are keyed by it (RowKey.block_limit).
M1/M2/L5/L6 contract section 19, module docstring and user guide: Otsu separation baseline
2/pi and correlation_1 as the emptiness marker, scale-free only under multiplication,
tenengrad = normalized_variance x (1 - rho_2), local_focus_p10 as content coverage,
central-difference Tenengrad and Pearson correlations.
L3 the QC sums' memory (qc_memory, band sized min(64, rows) + reach) is in the plane and block
estimates (plane_binning.Binning). L4 float planes are centred on their mean.
L7 manifest.py and run/pipeline.py split cleanly (manifest_files, run/state, run/outputs); the
docstring notes the squeeze removed are back.
F1 blank_reasons column. F2 block tolerances are observations, with a boundary test. F3 the
reader and writer refuse inconsistent rows. F4 reuse compares the full row identity.
F5 blocked-reader test with 16-row blocks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm
…ock; codec test limit

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm
…unt the QC working memory

The run's output code moved from pipeline to run.outputs, so the clock and the
histogram and manifest writers are patched there. The whole-plane estimate now
includes qc_memory; the block-read tests take their limit from block_limit, the
small plane's estimate with the QC term of one row of the large plane, so the
large plane is still read in blocks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm
@guijacquemet
guijacquemet marked this pull request as ready for review November 22, 2026 15:53
@guijacquemet
guijacquemet merged commit 6e47f3f into main Nov 22, 2026
8 checks passed
@guijacquemet
guijacquemet deleted the claude/image-qc-metrics branch November 22, 2026 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants