Skip to content

Fix Download.GetRange stopping after the first chunk - #339

Open
0z5a wants to merge 1 commit into
google:masterfrom
0z5a:fix/getrange-chunked
Open

0z5a wants to merge 1 commit into
google:masterfrom
0z5a:fix/getrange-chunked

Conversation

@0z5a

@0z5a 0z5a commented Nov 15, 2026

Copy link
Copy Markdown

Download.GetRange used end_byte both for the last byte of the requested range and for the last byte of each chunk request. After the first __ComputeEndByte call the range end was replaced by the chunk end, so the progress <= end_byte loop condition became false and the method returned after one chunk (two chunks when the total size was not known in advance). The result was a silently truncated stream whenever the range was larger than chunksize; with the default 1 MiB chunksize this affects any range over 1 MiB.

This change keeps the per-request end in a separate chunk_end variable, so the loop keeps going until the whole inclusive range has been fetched, as the docstring describes. Each request still uses a range of at most chunksize bytes, and use_chunks=False behaves as before.

The new testGetRangeChunked test downloads ranges larger than chunksize through a fake HTTP layer that honours the Range header (RFC 7233), with the total size known and unknown and with a negative start. It checks both the bytes written to the stream and the exact Range headers sent. It fails on master (only the first one or two chunks are requested) and passes with this change. All tests in apitools/base/py pass (227 on master, 228 with this change).

This change was drafted with Claude Code (claude-opus-5-5); the test results above are from the runs described.

GetRange used end_byte both for the end of the requested range and for
the end of each chunk request, so after the first request the range
end was replaced by the chunk end and the loop stopped. Ranges larger
than chunksize were silently truncated to one chunk (two when the
total size was unknown). Keep the per-request end in its own variable.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant