Skip to content

Fix o2-sim hang when the hit merger starts after the workers - #16048

Open
sawenzel wants to merge 2 commits into
AliceO2Group:devfrom
sawenzel:o2sim-worker-wait-for-driver
Open

sawenzel wants to merge 2 commits into
AliceO2Group:devfrom
sawenzel:o2sim-worker-wait-for-driver

Conversation

@sawenzel

@sawenzel sawenzel commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

This is fixing a bug where o2-sim hangs when the hit merger starts after the sim workers have finished. This was seen with --noGeant event generation in anchored MC, where a worker finishes within seconds. The worker exited 10 s after its last chunk with hits still queued in its ZMQ socket, so they were lost and the merger waited forever.

Workers now close their data channel with an infinite linger before exiting, which blocks until the merger has taken the hits. In addition, the merger is now spawned before the workers.

I tested this with the merger held back for 60 s before binding (20 events, 4 workers). dev hangs with 3 of 20 events written, and this PR finishes with all 20. Without the delay, runtime and kinematics are unchanged.

Assisted by Claude Code.

@sawenzel
sawenzel requested a review from a team as a code owner November 4, 2026 16:51
sawenzel and others added 2 commits November 5, 2026 20:44
This is fixing a bug where o2-sim hangs when the hit merger starts after the
sim workers have finished. A worker called _exit with hits still queued in its
simdata socket, so they were lost. The data channel now has an infinite linger
and the worker closes it before exiting, which blocks until the merger has
taken the hits. A parent-death signal makes sure a blocked worker does not
outlive the o2-sim process tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The merger is now listening when the workers send their first hits, so fast
workers (e.g. with --noGeant) no longer wait for it at exit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sawenzel sawenzel changed the title Keep o2-sim workers alive until the driver terminates them Fix o2-sim hang when the hit merger starts after the workers Nov 5, 2026
@sawenzel
sawenzel force-pushed the o2sim-worker-wait-for-driver branch from 9515f48 to 033437c Compare November 5, 2026 19:49

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant