You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Reduce first-waiter overhead in Semaphore permit handoff #400
On a fresh zero-permit semaphore, registering a pending request and handing it one permit took 232 ms versus 64 ms for Tokio over three million handoffs. The benchmark verifies that the waiter becomes ready and forgets the permit to return the semaphore to zero.
Apple M4 Max, macOS arm64, rustc 1.101.0-nightly. Each cell gives rank, elapsed time relative to Asyncband and the whole-batch median over 10 interleaved runs. Lower is faster. Supplemental measurements
Object lifetime explains much of the first row on this host. With the same three million handoffs, a fresh semaphore each round took 233 ms for Asyncband and 65 ms for Tokio. Reusing one across rounds gave 86 and 63 ms. Allocation counting found two allocations per fresh Asyncband handoff and zero for Tokio. After warm-up both reused instances had zero allocations per handoff. Asyncband's first waiter goes through the queue lock and arena. Tokio stores its waiter node inside the acquisition future. Tokio's pinned implementation
First contribution
Reproduce both lifetimes with the same zero-permit protocol, then isolate the queue's first allocation. A change should improve first registration and keep the reused case competitive. Check multi-permit ordering, cancellation after a partial or full grant, and absence of missed wakeups. PR #394 addresses acquisition-future size and is related work. PR #394
To run the supplemental peer_checks command below, save and run peer-checks.rb from the linked comment in a disposable checkout of 6102089. The benchmark and library sources match 36ba6f2. Generator and instructions
cargo x bench --bench primitives -- semaphore::acquire::handoff_permit --repeat 10 --raw
cargo x bench --bench peer_checks -- semaphore/ --repeat 10 --raw
Could you assign this to me? I'd run semaphore::acquire::handoff_permit on fresh and reused semaphores on an Apple M4 (10 cores, macOS 27.0.1), then repeat on Linux, counting allocations for each.
My guess is that the two allocations are the first lock of waiters on macOS and the first Arena slot. In a small standalone probe, a fresh Mutex allocated 64 bytes on first lock. That one should disappear on Linux, which would show how much of the gap is the queue.
If the queue does matter, an inline first slot would keep const construction but make every semaphore larger, while a node in the future would undo some of #394. I'd want your view on that before writing anything. I haven't run the benchmarks yet. I used Claude to help me read the code and write the probe.
Reproduced both lifetimes on Apple M5 (10 cores), macOS 27.0.1, rustc 1.99.1 stable, in a disposable checkout of 6102089 (PR #383 head, content-identical to the issue's 36ba6f2), with the issue's peer-checks.rb generator running the semaphore scenarios with --repeat 10.
Scenario
Rounds
Asyncband
Tokio
async-lock
semaphore/pending-handoff — a fresh semaphore per round
3,000,000
170 ms (147..480)
71 ms (63..163)
—
semaphore/try-one-and-return — one semaphore per repetition
30,000,000
243 ms (220..460)
193 ms (178..310)
261 ms (222..587)
semaphore/reused-handoff — the handoff round below, one semaphore per repetition
3,000,000
75 ms (69..117)
67 ms (60..93)
—
Medians over 10 interleaved runs; ranges are min..max. The first two rows are the semaphore/ scenarios of the generated peer_checks bench. The third row is a local adapter on top of the repository's ecosystem bench (patch below): the same pending-handoff round — acquire(1) polled to pending, release(1), polled to ready, forget() — with the semaphore constructed once per repetition instead of once per round, through the same interleaved runner against the same library sources, executed back-to-back with the peer bench in one session.
Allocation counting (a GlobalAlloc counter, 1,000,000 rounds; per-round counts are exact in every cell): a fresh Asyncband handoff performs two allocations per round against one for Tokio, and after warm-up both reused semaphores allocate nothing per handoff. Backtraces from the probe identify both sites (each recurs on every round):
64 B — the first lock of the waiters mutex heap-initializes the OS mutex: on this pthread-backed platform std::sync::Mutex boxes the OS mutex lazily (std::sys::sync::once_box::OnceBox in the trace; asyncband's internal::mutex::Mutex wraps std::sync::Mutex directly). Tokio's batch semaphore locks a std-backed mutex on the same path and pays the same boxing — this is its one fresh-round allocation.
192 B — Arena::insert grows the waiter slot vector from empty on the first registration, 4 × 48-byte slots (Arena<Node<WaitNode>>::insert reaching RawVec::grow_one, both under acquired_or_enqueue). This one is Asyncband-specific.
So the fresh-lifetime overhead is the queue's first-registration cost paid every round: the mutex's lazy boxing, shared with Tokio on this platform, plus the arena's first growth, which is Asyncband's extra allocation. Both disappear on a reused semaphore — the same handoff runs at 2.4x Tokio's time on a fresh semaphore and 1.12x on a reused one (the try/return row, which never queues a waiter, is 1.26x on a reused semaphore).
Adapter patch — benchmarks/benches/ecosystem/semaphore.rs plus the mod/register lines in main.rs
diff --git a/benchmarks/benches/ecosystem/main.rs b/benchmarks/benches/ecosystem/main.rs
index 561eedc..e9a7548 100644
--- a/benchmarks/benches/ecosystem/main.rs+++ b/benchmarks/benches/ecosystem/main.rs@@ -20,6 +20,7 @@ use benchmarks::harness::Runner;
mod broadcast;
mod mpmc;
mod mpsc;
+mod semaphore;
mod spmc;
mod waitgroup;
mod watch;
@@ -29,6 +30,7 @@ fn main() {
broadcast::register(&mut runner);
mpmc::register(&mut runner);
mpsc::register(&mut runner);
+ semaphore::register(&mut runner);
spmc::register(&mut runner);
waitgroup::register(&mut runner);
watch::register(&mut runner);
diff --git a/benchmarks/benches/ecosystem/semaphore.rs b/benchmarks/benches/ecosystem/semaphore.rs
new file mode 100644
index 0000000..b3c83c4
--- /dev/null+++ b/benchmarks/benches/ecosystem/semaphore.rs@@ -0,0 +1,75 @@+// Licensed to the Apache Software Foundation (ASF) under one+// or more contributor license agreements. See the NOTICE file+// distributed with this work for additional information+// regarding copyright ownership. The ASF licenses this file+// to you under the Apache License, Version 2.0 (the+// "License"); you may not use this file except in compliance+// with the License. You may obtain a copy of the License at+//+// http://www.apache.org/licenses/LICENSE-2.0+//+// Unless required by applicable law or agreed to in writing,+// software distributed under the License is distributed on an+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY+// KIND, either express or implied. See the License for the+// specific language governing permissions and limitations+// under the License.++use benchmarks::harness::Bencher;+use benchmarks::harness::Runner;+use benchmarks::support::bench_context;+use benchmarks::support::poll_pending;+use benchmarks::support::poll_pinned_ready;+use std::hint::black_box;+use std::pin::pin;++mod asyncband_reused {+ use super::*;++ pub(super) fn run(bencher: Bencher) {+ let semaphore = asyncband::semaphore::Semaphore::new(0);+ let mut context = bench_context();++ bencher.bench_local(|| {+ let mut acquire = pin!(semaphore.acquire(black_box(1)));+ poll_pending(acquire.as_mut(), &mut context);++ semaphore.release(black_box(1));+ let permit = poll_pinned_ready(acquire.as_mut(), &mut context);+ permit.forget();++ black_box(semaphore.available_permits())+ });+ }+}++mod tokio_reused {+ use super::*;++ pub(super) fn run(bencher: Bencher) {+ let semaphore = tokio::sync::Semaphore::new(0);+ let mut context = bench_context();++ bencher.bench_local(|| {+ let mut acquire = pin!(semaphore.acquire_many(black_box(1)));+ poll_pending(acquire.as_mut(), &mut context);++ semaphore.add_permits(black_box(1));+ let permit = poll_pinned_ready(acquire.as_mut(), &mut context).unwrap();+ permit.forget();++ black_box(semaphore.available_permits())+ });+ }+}++pub fn register(runner: &mut Runner) {+ runner.compare(+ "semaphore/reused-handoff",+ 3_000_000,+ &[+ ("Asyncband", &|b| asyncband_reused::run(b)),+ ("Tokio", &|b| tokio_reused::run(b)),+ ],+ );+}
Thanks to @nanami-0713 and @rokadepiyush49-rgb for profiling the allocation breakdown here and prototyping both inline storage designs in #419 and #415. Putting the findings together gives a clear answer on the trade-off:
Where the handoff_permit gap comes from: benchmarks/benches/primitives/semaphore/acquire.rs (handoff_permit) constructs and drops a fresh Semaphore::new(0) in every iteration. On macOS, each fresh handoff pays two one-time cold-start allocations: 64 B from std::sync::Mutex lazily boxing pthread_mutex_t on Apple targets (shared with Tokio on macOS, absent on futex targets, and tracked separately for waiter locks in Evaluate an inline lock for internal::mutex::Mutex on Apple targets #425), and 192 B from Arena's initial 4-slot Vec allocation on first contention. Once warmed up—which matches how Semaphore, Mutex, RwLock, Condvar, pools, and channels are used in practice—a reused Semaphore already allocates nothing and runs within ~1.12x of Tokio while keeping Acquire compact (24 B, perf(semaphore): shrink the permit acquisition future #394) and Unpin rather than embedding intrusive waiter nodes inside pinned futures.
Cost of inline first-waiter storage: Storing an inline slot alongside the Vec in Arena (perf(semaphore): register the first waiter without allocating #419) grows every Semaphore by +48 B on 64-bit (88 B -> 136 B), UpgradableRwLock by +96 B, dual-queue channels by +80 B, and permanently adds +48 B to every initialized OnceCell retained in a cache or OnceMap even when uncontended. Sharing storage via a two-mode enum in WaitList (perf(semaphore): avoid allocating queue storage for the first waiter #415) limits the size increase to +8 B per Semaphore (88 B -> 96 B), but its 31-scenario benchmark run shows steady-state regressions on reused handoff (1.03x) and 4–16 waiter contended queues (1.05x–1.11x across Semaphore, Mutex, OnceCell, and events) because every queue lookup and mutation pays a storage-mode branch.
Decision: Paying permanent struct growth and steady-state/contended throughput regressions across reusable synchronization primitives just to avoid the first allocation of a freshly constructed-and-dropped Semaphore::new(0) is not a net win. Workloads that genuinely do single-waiter one-shot signaling are better served by dedicated primitives (oneshot, Completion, or WaitGroup after perf(waitgroup): streamline completion for participants and observers #421).
Closing this as not planned for now. We can revisit if a design without steady-state or footprint regressions emerges, or split fresh vs. reused lifetimes in the benchmark suite separately if useful.
Activity
rokadepiyush49-rgb commentedon Nov 6, 2026
Could you assign this to me? I'd run
semaphore::acquire::handoff_permiton fresh and reused semaphores on an Apple M4 (10 cores, macOS 27.0.1), then repeat on Linux, counting allocations for each.My guess is that the two allocations are the first lock of
waiterson macOS and the firstArenaslot. In a small standalone probe, a freshMutexallocated 64 bytes on first lock. That one should disappear on Linux, which would show how much of the gap is the queue.If the queue does matter, an inline first slot would keep
constconstruction but make every semaphore larger, while a node in the future would undo some of #394. I'd want your view on that before writing anything. I haven't run the benchmarks yet. I used Claude to help me read the code and write the probe.tisonkun commentedon Nov 6, 2026
Thanks for your interest! We don't assign issues to individual contributors. Anyone is welcome to submit a PR directly for review.
nanami-0713 commentedon Nov 9, 2026
Reproduced both lifetimes on Apple M5 (10 cores), macOS 27.0.1, rustc 1.99.1 stable, in a disposable checkout of
6102089(PR #383 head, content-identical to the issue's36ba6f2), with the issue'speer-checks.rbgenerator running thesemaphorescenarios with--repeat 10.semaphore/pending-handoff— a fresh semaphore per roundsemaphore/try-one-and-return— one semaphore per repetitionsemaphore/reused-handoff— the handoff round below, one semaphore per repetitionMedians over 10 interleaved runs; ranges are min..max. The first two rows are the
semaphore/scenarios of the generatedpeer_checksbench. The third row is a local adapter on top of the repository's ecosystem bench (patch below): the samepending-handoffround —acquire(1)polled to pending,release(1), polled to ready,forget()— with the semaphore constructed once per repetition instead of once per round, through the same interleaved runner against the same library sources, executed back-to-back with the peer bench in one session.Allocation counting (a
GlobalAlloccounter, 1,000,000 rounds; per-round counts are exact in every cell): a fresh Asyncband handoff performs two allocations per round against one for Tokio, and after warm-up both reused semaphores allocate nothing per handoff. Backtraces from the probe identify both sites (each recurs on every round):waitersmutex heap-initializes the OS mutex: on this pthread-backed platformstd::sync::Mutexboxes the OS mutex lazily (std::sys::sync::once_box::OnceBoxin the trace; asyncband'sinternal::mutex::Mutexwrapsstd::sync::Mutexdirectly). Tokio's batch semaphore locks astd-backed mutex on the same path and pays the same boxing — this is its one fresh-round allocation.Arena::insertgrows the waiter slot vector from empty on the first registration, 4 × 48-byte slots (Arena<Node<WaitNode>>::insertreachingRawVec::grow_one, both underacquired_or_enqueue). This one is Asyncband-specific.So the fresh-lifetime overhead is the queue's first-registration cost paid every round: the mutex's lazy boxing, shared with Tokio on this platform, plus the arena's first growth, which is Asyncband's extra allocation. Both disappear on a reused semaphore — the same handoff runs at 2.4x Tokio's time on a fresh semaphore and 1.12x on a reused one (the try/return row, which never queues a waiter, is 1.26x on a reused semaphore).
Adapter patch —
benchmarks/benches/ecosystem/semaphore.rsplus themod/register lines inmain.rstisonkun commentedon Nov 14, 2026
Thanks to @nanami-0713 and @rokadepiyush49-rgb for profiling the allocation breakdown here and prototyping both inline storage designs in #419 and #415. Putting the findings together gives a clear answer on the trade-off:
handoff_permitgap comes from:benchmarks/benches/primitives/semaphore/acquire.rs(handoff_permit) constructs and drops a freshSemaphore::new(0)in every iteration. On macOS, each fresh handoff pays two one-time cold-start allocations:64 Bfromstd::sync::Mutexlazily boxingpthread_mutex_ton Apple targets (shared with Tokio on macOS, absent on futex targets, and tracked separately for waiter locks in Evaluate an inline lock for internal::mutex::Mutex on Apple targets #425), and192 BfromArena's initial 4-slotVecallocation on first contention. Once warmed up—which matches howSemaphore,Mutex,RwLock,Condvar, pools, and channels are used in practice—a reusedSemaphorealready allocates nothing and runs within ~1.12xof Tokio while keepingAcquirecompact (24 B, perf(semaphore): shrink the permit acquisition future #394) andUnpinrather than embedding intrusive waiter nodes inside pinned futures.VecinArena(perf(semaphore): register the first waiter without allocating #419) grows everySemaphoreby+48 Bon 64-bit (88 B -> 136 B),UpgradableRwLockby+96 B, dual-queue channels by+80 B, and permanently adds+48 Bto every initializedOnceCellretained in a cache orOnceMapeven when uncontended. Sharing storage via a two-modeenuminWaitList(perf(semaphore): avoid allocating queue storage for the first waiter #415) limits the size increase to+8 BperSemaphore(88 B -> 96 B), but its 31-scenario benchmark run shows steady-state regressions on reused handoff (1.03x) and 4–16 waiter contended queues (1.05x–1.11xacrossSemaphore,Mutex,OnceCell, and events) because every queue lookup and mutation pays a storage-mode branch.Semaphore::new(0)is not a net win. Workloads that genuinely do single-waiter one-shot signaling are better served by dedicated primitives (oneshot,Completion, orWaitGroupafter perf(waitgroup): streamline completion for participants and observers #421).Closing this as not planned for now. We can revisit if a design without steady-state or footprint regressions emerges, or split fresh vs. reused lifetimes in the benchmark suite separately if useful.