diff --git a/docs/parity/README.md b/docs/parity/README.md index 609688a..9f8df25 100644 --- a/docs/parity/README.md +++ b/docs/parity/README.md @@ -581,6 +581,70 @@ engine's corpus parse time (rs 6.5 s each of its 79 s total; ruby --- +## D. WO20 — parse-parity harness residual closure (2026-11-17, #285) + +`scripts/parse-parity.mjs` parses the whole corpus (maps/ + libs/) +with the ts parser (`parseIsc` + `expandDocument`) and the py parser +(`parse_isc`), projects both documents onto one canonical JSON schema, +and reports every structural difference with a kind, a path, and both +renderings. The audit baseline was **412/435 files identical** with +826 divergence rows in three kinds: item-kind 678, constraints 144, +item-value 4 — all attributed to the canonical projection, not to +either parser. WO20 closed all three classes by stating the shared +lowering each class collapses to; the harness now reports +**435/435, zero divergence kinds**. + +The reconciliation rules (each checkable against the cited code, so a +future reader can re-derive them): + +1. **Constraint written order → fixed kind order** (resolves the 144 + `constraints` rows plus 138 positional `item-kind` rows and the 4 + `item-value` rows — e.g. `alalc-ell-Grek-Latn-2010` writes + `after "Ν" before "Κ"`: ts preserves written order, while the py + SubRule model stores the four kinds as named fields, so the py + normalizer emitted them in fixed `before/after/not_before/ + not_after` order; comparing positionally swapped both the kinds + and the values). The four constraint kinds are simultaneous + lookarounds — written order between them carries no semantics — + so the canonical schema now projects BOTH parsers onto the fixed + kind order (stable within a kind). Genuine differences still + diverge: a one-sided kind lands at a different position or changes + the length; a duplicated kind (ts keeps every written copy, py + keeps the last) shows as a length or value mismatch. No corpus + rule writes a duplicated kind, so none of the 286 rows hid one. +2. **`none` → the empty string** (resolves 493 `item-kind` rows — + ts kept parse-level `{type:"none"}` where py lowers at parse). + Both runtimes lower identically at their compile layers: ts + `src/isc/converter.ts` `case "none"` → `StringItem("")` and + `src/runtime/compile-item.ts` `none: { re: "", literal: "" }`; py + `grammar/actions.py` ("converter parity: `none` lowers to the + empty string") → `StringItem("")`. py cannot represent `none` + distinct from `""`, so the canonical form is the shared lowering. +3. **`any_character` → the any_character stdlib alias** (resolves 47 + `item-kind` rows — ts `grammar/actions.ts` lowers the `any_char` + literal to a function atom, py to `AliasItem("any_character")`). + Both runtimes compile it through the alias path with the same + body: ts `src/isc/converter.ts` keeps the alias path for + `any_character` ("the primitive spelled as a function atom"), + `src/runtime/compile-item.ts` `any_character: { re: ".", + literal: "" }`; py `items.py` `ALIAS_COMPILED["any_character"]` + `re: "."`. The canonical form is that shared lowering. + +The py-vs-py swap driver (`interscript-py/scripts/parse_parity_py.py`) +already projected both of its parsers onto the same schema +(constraints re-sorted into the fixed kind order, `none` lowered); +its canonical judgment is unchanged by this closure, and its raw +projection (which includes the written constraint order) never +compared across grammars. + +Gates after the closure: harness 435/435 (zero kinds); py-vs-py +driver canonical 432/435, raw 432/435 (hand rev `7a84260`, +unchanged); py rs-golden zero-delta; ts parser-parity (289 maps) +green. No grammar (`.parg`) edits and no parser/model changes were +required — the residual was entirely in the canonical projection. + +--- + ## Reproducing ```sh diff --git a/scripts/parse-parity.mjs b/scripts/parse-parity.mjs index b6f4ad7..4e16ee9 100644 --- a/scripts/parse-parity.mjs +++ b/scripts/parse-parity.mjs @@ -14,10 +14,14 @@ // per-parser normalizer (the two IRs render the same source construct // differently by design — e.g. the Python parser lowers `none` to an empty // string item and lowers `to`-position function atoms to funcall-inline). -// Where a construct is representable in one parser but NOT the other, or -// renders under a different kind, the harness reports a divergence instead -// of force-passing: the first run is an AUDIT of the four hand-synced -// grammar copies. +// Where the canonical projection cannot equate a construct — a kind or +// value one parser derives and the other does not, a rule shape only one +// can represent — the harness reports a divergence instead of +// force-passing. Constructs BOTH runtimes lower identically at their +// compile layers (`none` -> "", `any_character` -> stdlib alias, +// constraint written order -> fixed kind order) are projected onto that +// shared lowering (WO20 #285: the audit's 826-row residual was exactly +// these three mechanical classes). // // Usage: // node scripts/parse-parity.mjs [--json ] [--verbose] @@ -264,7 +268,7 @@ function parseJsonl(stdout) { // --------------------------------------------------------------------------- // Canonical schema — shared target of the two per-parser normalizers // -// Items: {t:"s",v} | {t:"none"} | {t:"prim",name} | {t:"fn",name} +// Items: {t:"s",v} | {t:"prim",name} | {t:"fn",name} // | {t:"alias",name} | {t:"ref",id} | {t:"cap",inner} // | {t:"cat",parts} | {t:"any",of} | {t:"range",lo,hi} // | {t:"maybe",inner} | {t:"some",inner} @@ -283,9 +287,25 @@ function canonTsItem(it) { if (!it || typeof it !== "object") return { t: "shape", raw: String(it) } switch (it.type) { case "string": return { t: "s", v: it.value } - case "none": return { t: "none" } + case "none": + // WO20 (#285): `none` lowers to the empty string on both sides — + // ts at compile (converter.ts `case "none"` -> StringItem(""); + // compile-item.ts `none: { re: "", literal: "" }`), py at parse + // (actions.py "converter parity: `none` lowers to the empty + // string"). py cannot represent `none` distinct from "", so the + // canonical form is the shared lowering. + return { t: "s", v: "" } case "primitive": return { t: "prim", name: it.name } - case "function": return { t: "fn", name: it.name } + case "function": + // WO20 (#285): `any_character` is the any_character primitive + // spelled as a function atom (actions.ts lowers the any_char + // literal this way). Both runtimes compile it through the + // stdlib-alias path — ts converter.ts keeps the alias path + // (compile-item.ts `any_character: { re: ".", literal: "" }`), + // py represents it as AliasItem (items.py ALIAS_COMPILED + // `re: "."`). The canonical form is that shared lowering. + if (it.name === "any_character") return { t: "alias", name: it.name } + return { t: "fn", name: it.name } case "alias_ref": return { t: "alias", name: it.name } case "capture": return { t: "ref", id: it.index } case "capture_group": return { t: "cap", inner: canonTsItem(it.inner) } @@ -330,7 +350,9 @@ function canonTsRule(r) { return { from: canonTsItem(r.from), to: canonTsItem(r.to), - constraints: (r.constraints ?? []).map((c) => ({ kind: c.kind, item: canonTsItem(c.item) })), + constraints: orderConstraints( + (r.constraints ?? []).map((c) => ({ kind: c.kind, item: canonTsItem(c.item) })), + ), ...(r.when ? { when: [...r.when] } : {}), } } @@ -339,13 +361,28 @@ function canonTsRule(r) { // constraint kinds is lost, and a duplicated kind keeps only the last. const PY_CONSTRAINT_ORDER = ["before", "after", "not_before", "not_after"] +// WO20 (#285): the four constraint kinds are simultaneous lookarounds — +// written order between them carries no semantics, and the py SubRule +// model stores them as four named fields (written order lost, duplicated +// kind keeps the last). The canonical schema therefore projects BOTH +// parsers onto the fixed kind order (stable within a kind). A genuine +// difference still diverges: a kind present on one side only lands at a +// different position or changes the array length, and a duplicated kind +// (ts keeps every written copy, py keeps the last) shows as a length or +// value mismatch. +function orderConstraints(cs) { + return PY_CONSTRAINT_ORDER.flatMap((k) => cs.filter((c) => c.kind === k)) +} + function canonPyRule(r) { return { from: canonPyItem(r.from_), to: canonPyItem(r.to), - constraints: PY_CONSTRAINT_ORDER - .filter((k) => r[k] != null) - .map((k) => ({ kind: k, item: canonPyItem(r[k]) })), + constraints: orderConstraints( + PY_CONSTRAINT_ORDER + .filter((k) => r[k] != null) + .map((k) => ({ kind: k, item: canonPyItem(r[k]) })), + ), ...(r.when ? { when: [...r.when] } : {}), } }