You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
WO20: corpus coverage audit + conformance-depth batches from held sources #283
Corpus-wide test-coverage audit: rank every map whose embedded-test block is thin relative to its letter/rule table, then deepen the worst offenders with exhaustive per-row tests transcribed from the sources WE ALREADY HOLD (interscript-private-references: 92 ALA-LC docs, DIN, BSI). No new source acquisition.
Audit: per map, count embedded tests vs the table size its rules/notes imply; rank ascending. Output a ranked table in the PR.
For the top thin maps whose sources are locally held: transcribe the FULL printed table into scripts/row-sources/<system>.json (provenance fields per the vai/hye/zgh/chr pattern), run scripts/gen-row-tests.mjs, append the generated rows.
Anti-fabrication rules BINDING: every row comes from the printed PDF page; skipped rows documented with reasons; a failing row is a FINDING recorded in the PR (source-anchored) — never force green, never edit map rules in this round.
Gates: rs golden refresh (additive; zero lost/changed rows), maps-CI all gates, iscrs test-all green.
Batch PRs of ~5-8 maps each; do not bundle the whole audit into one PR. Branch prefix: wo20-depth-<group>.
WO20 milestone: corpus audit complete; batch 1 PR open.
Audit done corpus-wide (428 maps): embedded-test count vs encoded rule table, ranked ascending; machine-expanded CJK/jamo tables excluded as not letter-table-shaped. Full ranked table is in the PR body of batch 1.
Batch 1 (odni-cyrillic, 8 maps): #287 — all eight thin ODNI maps (1-2 embedded tests each, 60-92 rules) deepened from the held ICS 630-01 annexes: 314 rows transcribed from the printed tables (every value cell verified against the rendered page), all passing end-to-end. Findings recorded, maps untouched: mkd dze encoded as Latin S/s instead of Ѕ/ѕ; che plain-ц row has no map rule at all; che пӏ/цӏ key mismatches; footnotes 1/3 unimplemented; kaz я "Ya" case slip; tgk е note semantics + see-Russian cells. Gates: test-all 11,985/0, ts parse 435/435, golden delta additive (0 lost/0 changed; +770 uncaptured, recapture PR to follow after all batches).
Planned next batches (sources held): din-1460-2 thin subset (62 maps at 4 tests each; table artifact tools/din1460-2-table.json is the held-source extraction), alalc small-table batch (bul 0 tests, sat 1, lif 2, mni 4, aze-Cyrl 3, ben-2017 3, kat-Geok 5), alalc SEA batch (shn 1, tha 3, mya 6, bod 5, khm 10, lao 10), iso batch (rus-9, tha-11941).
Batch 2 (din1460, 8 maps): feat(WO20): exhaustive per-row tests for eight thin DIN 1460-2 maps (coverage depth, batch 2) #288 — the eight thinnest DIN 1460-2:2011 maps (4 tests each) deepened from the held standard PDF: 265 rows generated from tools/din1460-2-table.json, re-verified cell-by-cell against the printed grid rectangles (pymupdf find_tables). That reconciliation surfaced 6 findings, never forced green: four cells where the feat: add all 62 DIN 1460-2:2011-10 Cyrillic non-Slavic maps #242 extraction dropped the combining cedilla (ķ/ņ flattened to k/n — ab 37, az 38, ba 39+47), and two broken sub "к̢" "" empty-target rules in atv/az generated from a parenthesized-cell fragment ("(" recorded instead of "(ķ)"), so printed-key input is silently deleted. DIN's parenthesized optional-value cells transcribed paren-stripped; the pos-34 PUA glyph resolved as j-acute. Gates: test-all 11,936/0, ts 435/435, golden +689 additive (0 lost/0 changed).
Batch 3 (alalc-small, 6 maps): feat(WO20): exhaustive per-row tests for six thin ALA-LC small-table maps (coverage depth, batch 3) #289 — bul (0 tests!), sat, mni, aze-Cyrl, ben-2017, kat-Geok deepened from the held ALA-LC PDFs: 358 rows transcribed, 327 generated green, 31 findings. Biggest: aze-Cyrl has no rule at all for Ё/ё, Ц/ц, Щ/щ, Ъ/ъ, Ь/ь, Э/э, Ю/ю, Я/я (tesh-digraph values printed) plus Й→Y-vs-Ĭ and a dotless-ı candrabindu; ben-2017's values diverge from the printed 2017 table in 11 cells (they match the 1997-era scheme: ca→cha, ṛ→ṙ, ya→j̱a, ś→s, ṃ→ṁ...); sat lacks ᱱ→n entirely. Also recorded: alalc-lif is not deepenable — the held limbu.pdf carries no Limbu glyphs at all (blank glyph column in text layer and render), so row keys cannot be sourced from the printed page. Gates: test-all 12,263/0 combined tree, ts 435/435, golden +1013 additive.
Remaining thin maps: alalc SEA (shn/tha/mya/bod/khm/lao — held PDFs but shape-font text layers, heavy visual transcription), iso-rus-9 + iso-tha-11941 (held ISO PDFs), din-1460-2 remaining 44 maps (mechanical, same pattern as #288). Not held: bgnpcgn-fas-1956, icao-heb-9303, ses-ara-1930, bgnpcgn-arm-1981, bgnpcgn-aze-1993, var-mon-1930. The single rs golden-recapture PR covering all batches follows after the batch PRs settle.
Activity
ronaldtse commentedon Nov 16, 2026
WO20 milestone: corpus audit complete; batch 1 PR open.
Audit done corpus-wide (428 maps): embedded-test count vs encoded rule table, ranked ascending; machine-expanded CJK/jamo tables excluded as not letter-table-shaped. Full ranked table is in the PR body of batch 1.
Batch 1 (odni-cyrillic, 8 maps): #287 — all eight thin ODNI maps (1-2 embedded tests each, 60-92 rules) deepened from the held ICS 630-01 annexes: 314 rows transcribed from the printed tables (every value cell verified against the rendered page), all passing end-to-end. Findings recorded, maps untouched: mkd dze encoded as Latin S/s instead of Ѕ/ѕ; che plain-ц row has no map rule at all; che пӏ/цӏ key mismatches; footnotes 1/3 unimplemented; kaz я "Ya" case slip; tgk е note semantics + see-Russian cells. Gates: test-all 11,985/0, ts parse 435/435, golden delta additive (0 lost/0 changed; +770 uncaptured, recapture PR to follow after all batches).
Planned next batches (sources held): din-1460-2 thin subset (62 maps at 4 tests each; table artifact
tools/din1460-2-table.jsonis the held-source extraction), alalc small-table batch (bul 0 tests, sat 1, lif 2, mni 4, aze-Cyrl 3, ben-2017 3, kat-Geok 5), alalc SEA batch (shn 1, tha 3, mya 6, bod 5, khm 10, lao 10), iso batch (rus-9, tha-11941).Not-deepenable findings (sources not held): bgnpcgn-fas-1956 (0 tests), icao-heb-9303 (1), ses-ara-1930 (2), bgnpcgn-arm-1981 (2), bgnpcgn-aze-1993 (2), var-mon-1930 (1). bsi-kor PD 6505 remains scan/owner-gated.
ronaldtse commentedon Nov 16, 2026
WO20 milestone: batches 2 and 3 open.
tools/din1460-2-table.json, re-verified cell-by-cell against the printed grid rectangles (pymupdf find_tables). That reconciliation surfaced 6 findings, never forced green: four cells where the feat: add all 62 DIN 1460-2:2011-10 Cyrillic non-Slavic maps #242 extraction dropped the combining cedilla (ķ/ņ flattened to k/n — ab 37, az 38, ba 39+47), and two brokensub "к̢" ""empty-target rules in atv/az generated from a parenthesized-cell fragment ("(" recorded instead of "(ķ)"), so printed-key input is silently deleted. DIN's parenthesized optional-value cells transcribed paren-stripped; the pos-34 PUA glyph resolved as j-acute. Gates: test-all 11,936/0, ts 435/435, golden +689 additive (0 lost/0 changed).Remaining thin maps: alalc SEA (shn/tha/mya/bod/khm/lao — held PDFs but shape-font text layers, heavy visual transcription), iso-rus-9 + iso-tha-11941 (held ISO PDFs), din-1460-2 remaining 44 maps (mechanical, same pattern as #288). Not held: bgnpcgn-fas-1956, icao-heb-9303, ses-ara-1930, bgnpcgn-arm-1981, bgnpcgn-aze-1993, var-mon-1930. The single rs golden-recapture PR covering all batches follows after the batch PRs settle.