Repository navigation
perf(channels): optimize bounded MPMC and SPMC queues - #382
ariesdevil wants to merge 1 commit into
Conversation
94d174b to
4ea81c4
Compare
|
@ariesdevil You may update the branch to resolve the conflict with the main branch, and I will also adopt the new benchmark best practice that you can leverage. Once this pull request is clean, I will do another review. |
9600dab to
065940a
Compare
|
|
||
| ### Notable changes | ||
|
|
||
| * Bounded MPMC queues now panic when created with a capacity greater than 2,097,151. |
There was a problem hiding this comment.
Why do we now have this limitation?
There was a problem hiding this comment.
The ring's tail packs the slot index, the disconnect mark, the capacity that permits and grants hold, the queued-senders bit, and a lap count into one 64-bit word, so every exchange on it checks all of that state at once. That is what makes a send that stalled before its exchange fail if reservations changed meanwhile; the versioned counter this replaced could be fooled after 2^24 reservations. The index and the reserved count each need about log2(capacity) bits, leaving the rest for the lap count, and a stale exchange can only succeed after the lap count wraps. Capping capacity at 2^21 − 1 keeps at least 20 lap bits, so a wrap takes at least 2^40 sends. At 2^28 it would take about 2^34 sends, which a fast producer reaches within minutes while another thread is descheduled. Larger buffers can use mpmc::unbounded.
3b0752c to
a1284ba
Compare
Back bounded MPMC and SPMC queues with preallocated rings that track producer and consumer progress separately and keep each side's hot state on its own cache line. Cursors publish completed payload transfers and cache peer progress, with no per-slot stamps or packed capacity bits. MPMC reservations and detached grants belong to producer state under the producer lock, and waiters use stable IDs instead of pointers into pending futures. Receiver registration is separate from the consumer cursor, and wake callbacks and waker destruction run outside locks. SPMC keeps a specialized single-producer ring. Exact logical capacity, reservation ordering, cancellation, disconnection, and endpoint traits are unchanged. Unbounded queues keep the mutex backend, which bounded queues also use at capacity one and beyond the ring allocation budget, and bounded MPMC uses on targets without native 64-bit atomics. Copy CachePadded from crossbeam-utils and record the incorporated crossbeam portions in LICENSE. The benchmarks gain reserved-send and crossbeam-channel MPMC adapters, fixed-work 1P/8C, capacity-one, spinning, and SPMC scenarios, and reservation micro-benchmarks. Co-authored-by: tison <wander4096@gmail.com>
71d9757 to
e224174
Compare
Related: #298
Summary
Optimize bounded MPMC and SPMC queues with separate producer/consumer progress and cache-line isolation, while preserving exact logical capacity, reservation ordering, cancellation, disconnection, and endpoint traits. Unbounded queues retain their mutex-based backend.
Capacity-one queues and configurations beyond the ring allocation budget retain lazy storage. Bounded MPMC also uses that backend on targets without native 64-bit atomics. There is no new public capacity limit. These implementations are not lock-free.
Known limitation: the immediately-ready capacity-one async round trip regresses on Linux. A fixed-single-CPU follow-up measured a 44.9% increase in median elapsed time; details are below.
Design Notes
CachePaddedhelper. Crossbeam provenance is retained in source andLICENSE.Benchmarks
Reference library:
ae99c15(main). Candidate:a1284ba. Both use the same expanded benchmark harness and rustc 1.99.0. Values below are median fixed-work elapsed times; negative percentages are faster, not tail-latency claims.0,2,...,30(one thread per reported core). Revisions used independent build directories and immutable benchmark executables with verified hashes. The Linux matrix collected 414 implementation measurements.mpmc/bounded/tasks-2p2c-cap64-4wAsyncbandMpmcReservempmc/bounded/threads-2p2c-cap64AsyncbandMpmcReservempmc/bounded/threads-1p8c-cap64mpmc/bounded/threads-2p2c-cap1spmc/bounded/tasks-1p4c-cap64-4wspmc/bounded/tasks-1p4c-cap1024-4wThe earlier stamped-slot revision (
c766001) remains faster in several MPMC workloads. On Linux, this candidate takes 30.4% longer for the 2P2C async case and 155.2% longer for the 2P2C threaded case than that revision, while the 1P8C threaded case takes 60.7% less time. This PR does not claim to retain all of that revision's peak throughput.Regressions and limits
primitives::mpmc::bounded::send_then_recvbenchmark uses the lazy-storage fallback, not the ring. In the six-run single-CPU follow-up, main's median was 755.5 ms and the candidate's was 1095 ms for 24 million round trips: +44.9%. Main also exhibits a slower mode near 924 ms; the candidate remains slower than that mode. The regression is unresolved; async backend dispatch/future construction is a follow-up investigation, not an established cause.Validation
cargo x test,cargo x lint, andcargo x checkpassed locally, along with MPMC/SPMC ecosystem smoke checks.powerpc-unknown-linux-gnufallback run passed 33 tests with 2 ignored.cargo x test: 735 passed. Ecosystem and primitives builds and smoke checks passed for all three compared revisions. Nightly lint, the nightly feature matrix, and Miri were not run on that host because the required toolchain/tools were unavailable.