Skip to content

fix(worker): use selector event loop for ZeroMQ workers on Windows - #717

Open
Wayfarer545 wants to merge 1 commit into
taskiq-python:masterfrom
Wayfarer545:fix/524-zmq-windows-event-loop
Open

Wayfarer545 wants to merge 1 commit into
taskiq-python:masterfrom
Wayfarer545:fix/524-zmq-windows-event-loop

Conversation

@Wayfarer545

Copy link
Copy Markdown

Closes #524

On Windows, a worker with ZeroMQBroker crashes on its first receive: the default
ProactorEventLoop has no add_reader, which pyzmq needs unless tornado is installed.
ProcessManager keeps restarting the worker, so the main process looks healthy but no
tasks run. Setting an event loop policy in the app doesn't help, because the loop is
created before the broker is imported.

The worker now replaces ProactorEventLoop with SelectorEventLoop right after the
broker is imported, only on Windows and only for ZeroMQBroker.

This is narrower than the "pyzmq is installed" check discussed in the issue. pyzmq is
often a transitive dependency (e.g. via ipykernel), and the selector loop doesn't support
asyncio.create_subprocess_*, so that check would silently break subprocesses for users
of other brokers.

Limitations of the selector loop for ZeroMQ workers on Windows:

  • asyncio.create_subprocess_* isn't available in tasks.
  • An idle worker doesn't wake up on Ctrl+C. taskiq worker still exits immediately, but
    WORKER_SHUTDOWN handlers don't run.
  • Objects bound to the event loop while the broker module is imported stay bound to the
    replaced loop and fail with attached to a different loop.

Setups with tornado installed worked on ProactorEventLoop before, with a RuntimeWarning,
and now lose subprocess support and shutdown handlers on idle Ctrl+C too.

Tested with a real worker process on Windows (Python 3.10, 3.13 and 3.14) and on Linux
(Python 3.10). Not tested on macOS.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Worker with ZeroMQ does not work on Windows with default configuration

1 participant