Skip to content

Reuse verified connections for wide object writes - #536

Merged
matrixarkai merged 1 commit into
mainfrom
reuse-chunk-group-connections
Nov 9, 2026
Merged

matrixarkai merged 1 commit into
mainfrom
reuse-chunk-group-connections

Conversation

@matrixarkai

Copy link
Copy Markdown
Owner

Summary

  • retain up to 32 verified data-group connections for one wide object write
  • reuse those connections for data without enlarging the eight-entry persistent cache
  • add a 32-group connection regression test and a 51-process sixteen-group recovery workload
  • document the deterministic connection reduction and the measured local run

With default four-group concurrency, a cold 32-group write now establishes 32 data connections instead of 52.

Validation

  • cargo test -j1 --all-features --test object_writer_connections -- --test-threads=1 — 21 passed
  • cargo check -j1 --all-targets --all-features — passed
  • cargo test -j1 --all-features --lib -- --test-threads=1 — 726 passed, 33 ignored measurements
  • release WSL recovery workload — 51 processes, 16 data groups, 80 recovered objects, zero retries

The release workload wrote 64 initial 16 KiB objects at 15.2 objects/s and 16 post-failover objects at 32.1 objects/s, then verified all objects after total restart.

@matrixarkai
matrixarkai requested a review from bjmeetsfo November 9, 2026 18:12
@matrixarkai
matrixarkai merged commit fdd2095 into main Nov 9, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants