Skip to content

Phase 6 item 6.5: clazel-jpeg, a baseline JPEG decoder - #192

Draft
atiernan wants to merge 7 commits into
phase6/6.3-shapefrom
phase6/6.5-jpeg
Draft

atiernan wants to merge 7 commits into
phase6/6.3-shapefrom
phase6/6.5-jpeg

Conversation

@atiernan

Copy link
Copy Markdown
Member

Item 6.5 of Phase 6: clazel-jpeg, a baseline JPEG decoder on the 6.3 decoder shape. Tier stays 0 (target 1).

Done

  • Decoder: T.81 sequential Huffman (SOF0/SOF1, 8-bit), restart intervals, interleaved and non-interleaved scans, 1/3/4 components, libjpeg-turbo's jidctint IDCT and fancy upsampling, JFIF YCbCr, Adobe APP14 (inverted CMYK, YCCK), EXIF Orientation only. Progressive, arithmetic, lossless, 12-bit and 2-component files are refused with Unsupported. Listed choices in crates/clazel-jpeg/src/lib.rs.
  • Data: libjpeg-turbo 3.2.1 testorig/testimgint/testimgari/monkey12, and a 132-file corpus made by scripts/jpeg-reference.py, each with libjpeg-turbo's decoded output; xtask converter and generated tables.
  • Tests: 134 files, 2,065,288 samples, every sample exactly equal to libjpeg-turbo's (tolerance 0); one-unit changes to an IDCT or upsampling constant fail it. Chunking at sizes 1 to 4096 equal; 2,680 byte mutants never panic; 24 unit and 15 integration tests pass natively.
  • Kani (WSL, 0.68.0): 12 harnesses, 12 pass: bit reader, Huffman decoding over any table with the invariant, one bit of a block, the two IDCT passes (composed: no overflow for every i16 block and u16 quantizer), range limit over every i64, upsampling over every geometry up to 16x16, orientation, YCbCr over all 2^24 inputs, CMYK over all 2^32.
  • Miri: the 24 unit tests pass under Miri (none over 5 s).

Not done, honestly

  • Reference is libjpeg-turbo 2.1.5 (Ubuntu's, via ctypes), not the pinned 3.2.1: building the 3.2.1 sources was refused by the session's permission system. SOURCE.md and the README say so; regenerate with the script when it can be built. CMYK is compared after the same C*K/255 conversion on both sides (libjpeg gives CMYK), so that formula is not independently checked. ITU-T T.83 streams not used (the gap 6.1 recorded); zune-jpeg not added.
  • No Kani harness for Table::build, the marker state machine, the entropy pipeline or step: the model checker's memory grew without bound (it also took the shared WSL machine down), so those harnesses were removed rather than left unrun. Jpeg::state_inv being kept is argued, not machine-checked.
  • Conformance suites not completed under Miri: strides are set and sized from native timings with the Miri branch forced (0.02 s, 0.03 s, 0.01 s per test), but the shared WSL machine kept dying during the runs.
  • cargo xtask verify --changed not run to completion: it cannot work from a worktree in WSL (git pointer is a Windows path); CI's full verification has not run, so nothing is promoted.
  • Not benchmarked or optimised (one bit per call in entropy decoding); the work buffer is the worst case for the config; a stream without EOI never completes.

🤖 Generated with Claude Code

atiernan and others added 5 commits November 17, 2026 12:11
…ibjpeg-turbo's output (item 6.5)

libjpeg-turbo 3.2.1's testorig, testimgint, testimgari and monkey12 with
licences, and 132 files made by scripts/jpeg-reference.py (every sampling
ratio, restart intervals, scan layouts, gray, YCbCr, RGB, CMYK, YCCK, SOF1,
16-bit tables, EXIF orientations) with the output of the installed
libjpeg-turbo 2.1.5 for each, through ctypes. The pinned 3.2.1 could not be
built in this session: the references are 2.1.5's, and SOURCE.md says so.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
T.81 sequential Huffman decoding (SOF0, SOF1, restart intervals,
interleaved and single-component scans), libjpeg-turbo's jidctint and fancy
upsampling, JFIF and Adobe colour, EXIF orientation; resumable one byte at a
time over caller-owned buffers. Tier 0, target 1. Tested against
libjpeg-turbo's output: 134 files, every sample exact.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Leaf harnesses (bit reader, Huffman construction and decoding, IDCT, range
limit, upsampling, orientation, colour, one block bit) and the state
machine's (marker syntax, tables, frame and scan headers, restart markers,
one bit through the whole pipeline). Results are in the README once run.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
12 harnesses pass (bit reader, Huffman decoding over any table, block
decoder, IDCT passes, range limit, upsampling, orientation, colour). The
state-machine harnesses and Table::build did not finish in the model checker
and are removed; the README says so.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…d changed-crate CI)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI took 397 s under Miri for the chunking test (every 4th small file at
chunk sizes 1 and 64), over the 300 s limit for one test: Miri ran about
13,000 times a debug build here, not the 1,600 the strides assumed. It
now takes every 8th small file at chunk size 1, about a quarter. The
normal test job is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch is waiting to be deployed

1 waiting deployment
full-verification — 973524ce Waiting Nov 20, 2026 by atiernan via Approve the proofs #603
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant