Repository navigation
Expose the decoded global stream and extent reads - #18
Merged
Merged
Conversation
mahach666
force-pushed
the
feat/object-extents
branch
from
November 17, 2026 15:53
d2a7d7e to
27e56e2
Compare
OpenStream decodes the whole file once into a PlaStream snapshot: every block's payload decoded in block order (raw as they are, zero masks as their full decode, mode 2 as their token-stream output), with false entries of the scan skipped. A directory record's position and count name a byte extent of this stream — the object's data — so PlaStream.ReadExtent returns an object's bytes by record or by position and count. The position/count semantics are confirmed against the object directories of real files and documented on the record.
Two fixes to the decoded stream: - Blocks of the scan that fail to decode are skipped and contribute no bytes; every stream offset at or past the first skipped block is shifted by the missing bytes. The stream now tracks that boundary and exposes it as PlaStream.UnreliableFrom, and both extent reads report it: a read whose position is at or past the boundary, or that reaches into it, throws InvalidDataException instead of returning bytes of other objects. A zero-length extent at the boundary is shifted too. The record fits test of the sample corpus applies the exact-fit checks only to the records before the boundary and asserts the report for the records past it. - A record position wider than long (ulong.MaxValue) was cast before the bounds check and came out as an ArgumentOutOfRangeException from the extent read; the record read now compares against the stream length as unsigned and reports the documented InvalidDataException.
mahach666
force-pushed
the
feat/object-extents
branch
from
November 18, 2026 18:15
11b5195 to
c2ec387
Compare
Contributor
Author
|
All review points are fixed in c2ec387 (rebased on the updated feat/lz-chunks):
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #17 (uses the mode-2 decode; review and merge that first). Continues #7.
What this adds
PlaDocument.OpenStream()→PlaStream— the decoded global stream: every block's payload decoded in block order (raw as they are, zero masks as their full decode, mode 2 as their token-stream output). False entries of the scan are skipped. The stream is a snapshot: it holds the whole decoded file in memory and the document may be disposed afterwards.PlaStream.ReadExtent(...)— the bytes of one extent:ObjectDirectoryRecord— an object's data.InvalidDataExceptionwhen the record reaches past the stream;position, count— arbitrary range;ArgumentOutOfRangeExceptionon bad bounds.The position/count semantics of
ObjectDirectoryRecordare now documented (previously "units not documented"): position and count name a byte extent of the decoded global stream — the object's data.Evidence
minPos = 0— the addressing matches the real stream.EveryDirectoryRecordFitsTheGlobalStreamasserts this.Notes for review
src/PlaIO/PlaStream.csis the whole new surface;OpenStreamonPlaDocumentwraps the internal window decode.