Skip to content

Measure batched neighbor queue insertion - #1600

Draft
Wei Wu (wuw92) wants to merge 3 commits into
mainfrom
wuw92-beam-queue-benchmark
Draft

Wei Wu (wuw92) wants to merge 3 commits into
mainfrom
wuw92-beam-queue-benchmark

Conversation

@wuw92

@wuw92 Wei Wu (wuw92) commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Queue-only benchmarks for #1590. This PR is not intended to merge.

Results

The three-platform CI run shows gains across small, medium, and large queues. At L=2,000 with batches of 2,000-4,000 candidates, batch_insert is 4.88-23.11x faster than per-item insert, depending on the platform.

Performance depends on the workload. At L=20 with 40 candidates, batching is 23-34% slower across all three platforms. At L=2,000 with 8 candidates, it is 60% slower on Linux x64, 6% slower on Windows, and 1.52x faster on ARM64.

Queue insertion speedups across three CI platforms, including regressions

Speedup is median(insert) / median(batch_insert) over 27 samples per method (3 processes, 9 samples each); above 1x favors batching. Chart panels use independent x-axes. These shared-runner measurements isolate queue updates, not end-to-end search.

Raw JSON and full result tables (ZIP downloads; retained for 30 days):

Run

From this PR's checkout, run the same 23-case workload without a dataset (PowerShell):

cargo build --locked --release -p diskann-benchmark
1..3 | ForEach-Object {
  .\target\release\diskann-benchmark.exe run `
    --input-file .\diskann-benchmark\example\neighbor-queue.json `
    --output-file "run-$_.json"
}
python .github\scripts\summarize_neighbor_queue.py run-1.json run-2.json run-3.json

Edit neighbor-queue.json to change queue capacity, batch size, candidate placement, or cutoff survivors. The summary script reports batch_insert / insert, so values below 1 favor batching.

Wei Wu (wuw92) and others added 2 commits November 13, 2026 19:55
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@wuw92
Wei Wu (wuw92) force-pushed the wuw92-beam-queue-benchmark branch from 1387514 to ba0b492 Compare November 13, 2026 11:55
@wuw92
Wei Wu (wuw92) force-pushed the wuw92-beam-queue-batching branch from da80f17 to 25e8b68 Compare November 13, 2026 11:55
@wuw92 Wei Wu (wuw92) changed the title Benchmark neighbor queue append vs insert Measure batched neighbor queue insertion Nov 13, 2026
@wuw92
Wei Wu (wuw92) changed the base branch from wuw92-beam-queue-batching to main November 13, 2026 12:47
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.04991% with 52 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.14%. Comparing base (dd896a3) to head (a58bd09).

Files with missing lines Patch % Lines
diskann-benchmark/src/neighbor_queue.rs 86.05% 52 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1600      +/-   ##
==========================================
- Coverage   92.14%   92.14%   -0.01%     
==========================================
  Files         621      622       +1     
  Lines      130986   131546     +560     
==========================================
+ Hits       120694   121207     +513     
- Misses      10292    10339      +47     
Flag Coverage Δ
miri 92.14% <91.04%> (-0.01%) ⬇️
unittests 92.09% <91.04%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
diskann-benchmark/src/main.rs 94.08% <100.00%> (+0.01%) ⬆️
diskann/src/graph/index.rs 96.47% <100.00%> (-0.01%) ⬇️
diskann/src/graph/search/inline_filter_search.rs 99.52% <100.00%> (-0.01%) ⬇️
diskann/src/graph/search/multihop_filter_search.rs 100.00% <100.00%> (ø)
diskann/src/neighbor/queue.rs 99.13% <100.00%> (+0.19%) ⬆️
diskann-benchmark/src/neighbor_queue.rs 86.05% <86.05%> (ø)

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants