Repository navigation
Replay the pinned 675-row unicode generation on the installed wheel - #3
Draft
suleman-uzair wants to merge 4 commits into
Draft
suleman-uzair wants to merge 4 commits into
suleman-uzair wants to merge 4 commits into
Conversation
added 4 commits
November 13, 2026 14:59
scripts/corpus_replay.py checks out the plurimath-testsuite commit named by ci/testsuite-pin.yml, selects the pinned ledger entry, recomputes the snapshot digest, verifies every payload's bytes, SHA-256 and row IDs against provenance, and only then replays the rows: raw-tree rows through parse_legacy_raw with full tree equality, typed-model rows through parse_model_json for their outcome. The executed IDs must match ci/testsuite-replay.json. Offline tests prove the tamper refusals. A replay job runs it on Linux, macOS and Windows.
ci/testsuite-pin.yml pins plurimath-testsuite acbdf03 (merged in #50), unicode generation 5, sha256:282fb3ee... It holds the 72 generation-4 rows and the 675 unicode-external rows at both the raw-tree and the typed-model boundary. ci/testsuite-replay.json records the 747 + 747 rows this consumer executes and the SHA-256 of their sorted IDs. Rows nest up to 170 levels, so the replay raises the recursion limit instead of dropping deep rows. It writes the executed IDs so the replay job can run the testsuite's validate-stages.rb --pin and validate-unicode-external.rb --executed checks.
The typed-model boundary only checked accepted or refused. It now compares each model row's field names, order and values with the codec JSON the wheel returns, and reports the first differing path per row.
…6d83b22 Add opt-in parse_model_legacy_json (umath_parse_model_legacy_json_v1) beside the unchanged Strict parse_model_json. The typed replay compares refusal caller byte offsets and drops only writer-omitted empty text and empty lists, with negative controls for the rule. Repin umath-rs, Cargo.lock hash and parsanol SDK.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replays unicode generation 5 of plurimath-testsuite on the installed platform wheel. That generation holds the 675 UnicodeMath test rows imported from UnicodeMathML (MIT, Noah Doersing 2019).
scripts/corpus_replay.pychecks out the commit named inci/testsuite-pin.ymland selects that ledger entry, not the latest one. It recomputes the snapshot digest and checks every payload's bytes, SHA-256 and row IDs againstprovenance.yaml. Only after those checks pass does it run any row.parse_legacy_raw. The preprocessed text and the whole raw tree must be equal, or for a refusal, the grammar byte offset.parse_model_jsonand are checked for outcome only (accepted or refused). The Ruby LML Document and the Rust model JSON are different shapes, so this PR claims no tree equality for them.No row is skipped.
ci/testsuite-replay.jsonrecords the rows executed per boundary: 747 raw-tree and 747 typed-model, which is the 72 generation-4 rows plus the 675 external rows. It also records the SHA-256 of the sorted IDs. The replay fails if any row disagrees or if the executed set differs from that record. Rows nest up to 170 levels, so the replay raises the recursion limit instead of dropping them.The new
replayjob runs on Linux, macOS and Windows with Python 3.14, the newest Python with PyYAML libyaml wheels on every platform. After the replay, it runs the testsuite's ownvalidate-stages.rb --pinandvalidate-unicode-external.rb --generation 5 --executed. The offline tests inscripts/tests/test_corpus_replay.pyshow that a tampered payload, an unlisted file, the wrong commit, generation or digest, a dropped row and an unsafe path are all refused.Pinned testsuite commit: acbdf0319dfb9bc49c1726bdbdedac276476e9b5 (merged via plurimath/plurimath-testsuite#50).