Skip to content

Add fail-closed gameplay evidence validator - #17

Open
ahrazzle wants to merge 2 commits into
feat/formal-verification-loop-v0.2from
feat/gameplay-evidence-exchange
Open

ahrazzle wants to merge 2 commits into
feat/formal-verification-loop-v0.2from
feat/gameplay-evidence-exchange

Conversation

@ahrazzle

Copy link
Copy Markdown
Member

Summary

  • Implement the gameplay-evidence/v0.1 schema and fail-closed validator.
  • Verify canonical JSON, action and trajectory commitments, negative controls, and producer/consumer provenance.
  • Separate producer assertions from independently admitted consumer badges.
  • Add mutation coverage for tampered payloads, actions, trajectories, checks, labels, and commitments.

Evidence boundary

Structural validation is not live execution or model proof. Independent structural admission is limited to OBSERVED; producer REPLAYED assertions are retained only as source evidence. A replay or fixture never promotes an artifact to LIVE_CONFIRMED.

The cross-repository acceptance fixtures and exact digests are recorded in the accompanying implementation commit and documentation. RMMZ fixture artifact SHA-256: c852b43bfa6c1279148625434af5119a4aa5767418e6e742ee97b645bf5a5f1d. Soko Heroes rr artifact SHA-256: 0a310102c655880e587ff3502600daedcd868a4a93578061891ee28eef78585b.

Verification

At commit ee2c2439160d5cee50098a397f7a11ca4f06aea9:

python -m pytest tests/test_gameplay_evidence.py -q

The evidence test suite passed. Cross-consumer checks accepted both pinned fixtures with admitted_badges limited to OBSERVED; resealed trajectory tampering failed closed through payload-commitment mismatches.

@zero-aska
zero-aska changed the base branch from main to feat/formal-verification-loop-v0.2 November 22, 2026 17:14

@abu-aska abu-aska left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The isolated gameplay-evidence commit ee2c2439 (4 files, +1,163/-0) is clean, zero-dependency, and passes 32/32 OK in tests/test_gameplay_evidence.py across -I -B, -I -O -B, and -I -B -W error (with zero bare asserts; rejects float 1.0/-0.0, NaN/Inf, duplicate JSON keys, int64 overflow, passed=1, cyclic objects, and enforces the mixed PASS+FAIL veto).

However, four concrete blocking issues (branch stacking, failing CI, recursion DoS, and surrogate/cross-repo parity bugs) should be resolved before merge:

  1. Branch stacked on an older commit of PR #16 (463 files +137,882/-5 / 48 commits vs. main @ 575286f0):
    • ee2c2439's parent is d8e92d84201a (one commit behind open PR #16 @ 8695c9e9, which itself builds on #6/#7/#15), pulling 459 unrelated files (47 commits, including +110 lines in .github/workflows/ci.yml) into PR #17.
    • Please rebase/cherry-pick ee2c2439 cleanly onto main (or hold until #6/#16 land) so the PR diff contains only the 4 gameplay-evidence files.
  2. Failing CI (standalone (ubuntu-24.04) & standalone (windows-2022) in run 45046280747):
    • Both runners fail python -m mcproof.prose_gate with 3 findings on docs/GAMEPLAY_EVIDENCE.md (reproduced locally, rc=1):
      • docs/GAMEPLAY_EVIDENCE.md:28: em dash (T2) (envelope—not)
      • docs/GAMEPLAY_EVIDENCE.md:30: semicolon stitching clauses (T2)
      • docs/GAMEPLAY_EVIDENCE.md: paragraph over 420 chars on lines [16, 28].
    • (Confirmed locally: verification/leak_audit.py passes 286 files and verification/build_final_receipt.py leaves verification/VERIFICATION_RECEIPT.json clean.)
  3. Uncaught RecursionError DoS crash on deeply nested JSON (src/mcproof/gameplay_evidence.py:64-91, 203-233):
    • _strict_json_tree and _validate_canonical_value recurse with no depth or size limit.
    • Reproduced live: a 1,200-deep nested array in payload.outcome passes load_gameplay_artifact and then crashes verify_gameplay_artifact with an unhandled RecursionError: maximum recursion depth exceeded (contrast warp-drive#36, which validates iteratively with GateLimits(max_depth=32) and returns envelope_valid=False). Convert traversal to an explicit stack with depth/size ceilings.
  4. Unpaired UTF-16 surrogates silently accepted in envelope fields (src/mcproof/gameplay_evidence.py:66, 101, 205):
    • UTF-8 encodability is checked only on source["raw"] (L366), while canonical_json_bytes uses json.dumps(..., ensure_ascii=True) (L101), silently escaping unpaired surrogates ("\ud800" -> \ud800).
    • Reproduced live: an artifact with "claim.statement": "bad \ud800 surrogate" returns valid=True, source_badges=['OBSERVED', 'REPLAYED'] here, while warp-drive#36 rejects it (INVALID_UNICODE). Validate .encode("utf-8") on every string and object key in _strict_json_tree and _validate_canonical_value.

Cross-Repo Parity & Minor Hygiene Notes

  • Cross-repo gameplay-evidence/v0.1 contract reconciliation with Moissanite-Labs/warp-drive#36:
    • payload.model and payload.initial_state: mcproof#17 (L274-284, schema:68,70) accept any jsonValue ([1,2,3]), while warp-drive#36 (L397-399) requires dict.
    • source.raw == "": accepted here (L336, schema:115), rejected in warp-drive#36 (L450).
    • mechanics[*].path: accepted here for CORE.YAML (rejected in warp-drive#36); also both validators currently accept bare ".yaml", newlines ("mechanics/a\nb.yaml"), and Windows drive prefixes ("C:foo.yaml").
    • negative_controls[*].mutation/.observed and identifier regex (^[A-Za-z0-9_.:-]+$): properly enforced here, currently missing/looser in warp-drive#36.
  • Minor input & fixture precision: REPOSITORY_RE (L58) matches "../..", COMMIT_RE (L56) accepts "0" * 40, raw JSON -0 parses as int(0) via json.loads (L130), and the PR body states the two pinned acceptance fixture digests (c852b43bfa6c..., 0a310102c655...) "are recorded in the accompanying implementation commit and documentation", but neither digest appears in the 4 committed files of ee2c2439.

Independent clean-room review by agentic team with human oversight (aletheia v2.0.0, live CI & adversarial repro).

@abu-aska abu-aska left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean-room re-verification of 51d241333194 (base feat/formal-verification-loop-v0.2 @ c2ed9ca5, 4 files +1,167/-0, 0 workflow changes, 6/6 CI green).

Blockers Closed Since ee2c2439

  • Blocker #1 (branch stacking) — CLOSED: PR diff is now cleanly scoped to the 4 gameplay-evidence files (docs/GAMEPLAY_EVIDENCE.md, schema/gameplay-evidence-v0.1.schema.json, src/mcproof/gameplay_evidence.py, tests/test_gameplay_evidence.py, +1,167/-0, 0 .github/ changes).
  • Blocker #2 (CI / mcproof.prose_gate) — CLOSED: Confirmed locally across -B, -O -B, and -B -W error: python3 -m mcproof.prose_gate docs/GAMEPLAY_EVIDENCE.md prints check-prose: clean (1 file(s), 34 lines). tests/test_gameplay_evidence.py passes 32/32 OK across all 3 standard modes (-B, -O -B, -B -W error) and all 3 isolated modes (-I -B, -I -O -B, -I -B -W error), and aletheia v2.0.0 audit-check --strict-ancestors --safety-scan passes on all 4/4 files (0 findings).

Remaining Open Blockers (Unchanged Since ee2c2439, src/mcproof/gameplay_evidence.py Untouched)

Commit 51d24133 updated only docs/GAMEPLAY_EVIDENCE.md, so both runtime fail-closed blockers in src/mcproof/gameplay_evidence.py still reproduce live:

  1. Uncaught RecursionError DoS on deeply nested JSON (src/mcproof/gameplay_evidence.py:64-91, 203-233):
    • Re-verified live on 51d24133: a 1,200-deep nested array in payload.outcome passes load_gameplay_artifact(), then crashes both verify_gameplay_artifact() and canonical_json_bytes() with RecursionError: maximum recursion depth exceeded. Replacing recursion in _strict_json_tree and _validate_canonical_value with an explicit stack + depth/node ceiling closes this cleanly.
  2. Unpaired UTF-16 surrogates silently accepted (src/mcproof/gameplay_evidence.py:66, 101, 205):
    • Re-verified live on 51d24133: setting claim.statement = "Bad surrogate \ud800 here" (both direct dict and JSON \ud800 via load_gameplay_artifact) returns valid=True, source_badges=['OBSERVED', 'REPLAYED'], admitted_badges=['OBSERVED'], errors=[], and an unpaired surrogate in an object key ({"bad\ud800key": True} in payload.outcome) also returns valid=True, admitted_badges=['OBSERVED']. Checking .encode("utf-8") on every string value and object key in _strict_json_tree and _validate_canonical_value closes this cleanly and restores parity with warp-drive#36.

Keeping the prior CHANGES_REQUESTED active for Blockers #3 and #4 — ready to approve immediately once those two runtime guards and regression tests land!


Independent clean-room re-verification by agentic team with human oversight (aletheia v2.0.0).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants