Skip to content

Fix ThreadPrefetchDatasetIterator.start_prefetch starting a background thread when prefetch_buffer_size=0. - #1480

Merged
copybara-service[bot] merged 1 commit into
mainfrom
test_1023916265
Nov 21, 2026
Merged

copybara-service[bot] merged 1 commit into
mainfrom
test_1023916265

Conversation

@copybara-service

@copybara-service copybara-service Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Fix ThreadPrefetchDatasetIterator.start_prefetch starting a background thread when prefetch_buffer_size=0.

Previously, calling start_prefetch() with prefetch_buffer_size=0 started a prefetch thread on an unbounded queue while __next__ also read directly from the parent iterator, which could race and yield duplicate elements. Add a unit test verifying no prefetch thread is started when prefetch_buffer_size=0.

Also make test_early_break_continues_prefetching deterministic in prefetch_test.py: replace wall-clock sleeps with threading.Event synchronization (re-enabling the test on macOS), move it to PrefetchIterDatasetTest, and add test factory hooks on _ThreadPrefetchIterDatasetTestBase.


📚 Documentation preview 📚: https://google-grain--1480.org.readthedocs.build/

@copybara-service
copybara-service Bot force-pushed the test_1023916265 branch 2 times, most recently from e6e27f0 to b4d7488 Compare November 21, 2026 06:51
…und thread when `prefetch_buffer_size=0`.

Previously, calling `start_prefetch()` with `prefetch_buffer_size=0` started a prefetch thread on an unbounded queue while `__next__` also read directly from the parent iterator, which could race and yield duplicate elements. Add a unit test verifying no prefetch thread is started when `prefetch_buffer_size=0`.

Also make `test_early_break_continues_prefetching` deterministic in `prefetch_test.py`: replace wall-clock sleeps with `threading.Event` synchronization (re-enabling the test on macOS), move it to `PrefetchIterDatasetTest`, and add test factory hooks on `_ThreadPrefetchIterDatasetTestBase`.

PiperOrigin-RevId: 1027001944
@copybara-service
copybara-service Bot merged commit fb74613 into main Nov 21, 2026
4 of 5 checks passed
@copybara-service
copybara-service Bot deleted the test_1023916265 branch November 21, 2026 07:20

This branch was successfully deployed

1 active deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant