Repository navigation
Fix ThreadPrefetchDatasetIterator.start_prefetch starting a background thread when prefetch_buffer_size=0. - #1480
Merged
Merged
Conversation
copybara-service
Bot
force-pushed
the
test_1023916265
branch
2 times, most recently
from
November 21, 2026 06:51
e6e27f0 to
b4d7488
Compare
…und thread when `prefetch_buffer_size=0`. Previously, calling `start_prefetch()` with `prefetch_buffer_size=0` started a prefetch thread on an unbounded queue while `__next__` also read directly from the parent iterator, which could race and yield duplicate elements. Add a unit test verifying no prefetch thread is started when `prefetch_buffer_size=0`. Also make `test_early_break_continues_prefetching` deterministic in `prefetch_test.py`: replace wall-clock sleeps with `threading.Event` synchronization (re-enabling the test on macOS), move it to `PrefetchIterDatasetTest`, and add test factory hooks on `_ThreadPrefetchIterDatasetTestBase`. PiperOrigin-RevId: 1027001944
copybara-service
Bot
force-pushed
the
test_1023916265
branch
from
November 21, 2026 07:20
b4d7488 to
fb74613
Compare
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix
ThreadPrefetchDatasetIterator.start_prefetchstarting a background thread whenprefetch_buffer_size=0.Previously, calling
start_prefetch()withprefetch_buffer_size=0started a prefetch thread on an unbounded queue while__next__also read directly from the parent iterator, which could race and yield duplicate elements. Add a unit test verifying no prefetch thread is started whenprefetch_buffer_size=0.Also make
test_early_break_continues_prefetchingdeterministic inprefetch_test.py: replace wall-clock sleeps withthreading.Eventsynchronization (re-enabling the test on macOS), move it toPrefetchIterDatasetTest, and add test factory hooks on_ThreadPrefetchIterDatasetTestBase.📚 Documentation preview 📚: https://google-grain--1480.org.readthedocs.build/