Mapping-set selection and rc.2 x-oold-context transform by simontaurus · Pull Request #154 · OO-LD/oold-python

simontaurus · 2026-09-12T08:36:11Z

Changes

oold.utils.mappings (new): promote, mapping_sets, chain, context_of, synonym_entries, inline_scoped. This is the profile-based selection OOLD-EXT-4966 describes; it complements validation.context_resolution.promoted_terms, which deliberately never chooses a synonym (OOLD-EXT-8f62). Ported from oold-reference-schemas/scripts/{_shared,build_docs}.py and verified equivalent against that implementation on every module, all schemas, all three mapping sets plus consensus. Two deliberate extensions over the port: mapping_set_id may be a list (the spec allows it), and a @reverse synonym promotes to @reverse rather than an invalid @id+@reverse pair.
oold.utils.transform (replaced): to_rdf / from_rdf / transform over x-oold-context. Export promotes a named set; import rewrites every exactMatch synonym onto the primary IRI (no set needed), inverts @reverse relations, frames @graph input via validation.frame.schema_to_frame, and bridges document shapes when a promoted fragment carries @nest and a set is named. Readings A/B/C/E of Mapping-set selection can bridge document shapes via @nest, and the spec should say so oold-schema#135 are reproduced exactly and pinned as regression tests, including the boundary that vocabulary is bridged without a selection and shape is not.
tests/test_transform.py rewritten: the name* alias cases are re-encoded as x-oold-context with the same inputs and expected outputs, so the change of notation provably does not change what the transformation produces.
promote kept a term's scoped @context. A @reverse synonym dropped it, so an embedded object fell back to the consensus terms under a selected set. Direction does not change which terms the object at the end of an edge may use.

Notation

x-oold-context replaces the name* alias notation as what transform reads.

That notation predates the specification and was never part of it. Nothing in the spec surface changes. The commit stays feat:.

jsonld_to_jsonld and json_to_json are kept, deprecated, with their tests. They are the only readers of the old notation left.

Generic tools ported from oold-reference-schemas

Nothing ported assumes a directory layout, a module name or a docs pipeline.

mappings.inline_scoped and the ScopeSource protocol: resolves a term's scoped @context from the tree and promotes it with the same mapping set. Identical to build_docs.inline_scoped on 58 schema and mapping-set combinations.
oold.utils.locator: Locator plus MirrorRule, which states the published-IRI-to-path layout as data. term_chain matches _shared.chain on every schema. The cycle guard is new, the source recurses forever on one.
oold.validation.effective_views: the round trip per mapping set, with the URDNA2015 fallback for a reading that legitimately changes the document shape. Same verdict and triple count as scripts/effective_views.py on 26 readings. validation/compliance.py only ever exercises the declared context, so a defect confined to one set was invisible until now.
oold.utils.readings: Turtle and Mermaid per mapping set, returned rather than written. 54 readings byte-identical to build_docs.write_rdf.
validation.generate.synth_instance: an instance built from a schema's own examples and defaults. Identical on all 15 schemas and to the committed generated instances. inherited needed no port, mappings.chain over a Locator reproduces it exactly.
oold.utils.sssom: SSSOM sets and the crosswalk between them. 10 sets identical to build_mappings.py, except 51 rows where the source holds a literal 0x01 byte in place of a back-reference (build_mappings.py line 209) and emits a relative subject. The port writes what the comment beside it says.
oold.utils.labels: label harvest from pinned ontology snapshots. mapped_iris identical at 121 IRIs, and the harvest over the 16 pinned snapshots reproduces generated/labels.json byte for byte.

Left out: scripts/level.py, because OO-LD/oold-reference-schemas#19 changes surface() and compare(), the two functions a port would consist of.

Verification

923 passed, 10 skipped, 1 xfailed on top of current main, against 804, 10, 1 on main itself. make check green.
Rebased onto main after fix(frame): reverse IRI grouping, $ref embeds and default type anchors #179. The tests pass against the new schema_to_frame output rather than around it.
First consumer: schema-playground in https://github.com/OO-LD/oold-playgrounds/tree/feat/schema-playground vendors exactly these modules pending this PR; once released they can be deleted there.

Closes the mappings/transform part of the migration listed in https://github.com/OO-LD/oold-playgrounds/blob/feat/schema-playground/schema-playground/README.md#pending-upstream-migration

github-actions · 2026-09-12T08:36:32Z

Release preview

Merging this PR would release v2.4.0 (current: v2.3.1).

Changelog preview (truncated)

## v2.4.0 (2026-11-15)

### Bug Fixes

- **mappings**: Keep a promoted term's scoped @context
  ([`e87f580`](https://github.com/OO-LD/oold-python/commit/e87f580df7256a014bde51e60047e9d6afc63664))

### Features

- **generate**: Build an instance from authored examples and defaults
  ([`e23d4d3`](https://github.com/OO-LD/oold-python/commit/e23d4d31eed58dbad0ed43b48652a5ade1e047cc))

- **labels**: Harvest labels from pinned ontology snapshots
  ([`f1a7b67`](https://github.com/OO-LD/oold-python/commit/f1a7b67086e8a18aa52ced1dfc5871df6ff6558c))

- **locator**: Resolve a published IRI in the working tree
  ([`e085814`](https://github.com/OO-LD/oold-python/commit/e085814b68c0021b2b7e3fae4f3ab3fd9cf9adde))

- **mappings**: Inline a scoped @context under the selected set
  ([`bc6a90d`](https://github.com/OO-LD/oold-python/commit/bc6a90d96221c107739020381a0c8faf549aed9b))

- **readings**: Render an instance as RDF per mapping set
  ([`d6009ab`](https://github.com/OO-LD/oold-python/commit/d6009ab4013feff9151ae3ae14954946a5c08cf1))

- **sssom**: Derive mapping sets from the schemas that carry them
  ([`73e88db`](https://github.com/OO-LD/oold-python/commit/73e88db9e6cf96d04f1da8ca4211fae8b9fe6a03))

- **utils**: Mapping-set selection and rc.2 x-oold-context transform
  ([`dfeb43f`](https://github.com/OO-LD/oold-python/commit/dfeb43febec2377b7dc3be65115fdebab023b80d))

- **utils**: Rewrite synonyms in JSON-LD rather than in RDF
  ([`5b1f25d`](https://github.com/OO-LD/oold-python/commit/5b1f25d0c2020d948e16b759e816bc0331fffc90))

- **validation**: Round-trip an instance under every mapping set
  ([`cd3f8a5`](https://github.com/OO-LD/oold-python/commit/cd3f8a551e450620a3e7d991da0508c90dc39c09))

_{Preview via python-semantic-release and conventional commits.}

github-actions · 2026-09-12T08:37:31Z

📊 Benchmark Results

Click to see benchmark comparison

📊 Benchmark Comparison (threshold: 1.3x)
============================================================

⚠️  Performance Regressions:
  ❌ test_complex_graph: 0.0011s → 0.0017s (+55.3%, ratio: 1.55x)

✅ Performance Improvements:
  ✅ test_subclass_inheritance: 0.0543s → 0.0405s (-25.3%, ratio: 0.75x)

➖ Unchanged (within threshold):
  ➖ test_simple_dict_document_store: 0.0012s → 0.0012s (+0.7%)
  ➖ test_sqlite_document_store: 0.0013s → 0.0013s (+0.8%)
  ➖ test_local_sparql_store: 0.0289s → 0.0286s (-1.2%)
  ➖ test_oneof_subschema: 0.0438s → 0.0465s (+6.2%)
  ➖ test_enum_docstrings: 0.0402s → 0.0441s (+9.7%)
  ➖ test_class_hierarchy: 0.0372s → 0.0441s (+18.5%)
  ➖ test_core[v1]: 0.0279s → 0.0348s (+24.7%)
  ➖ test_core[v2]: 0.0328s → 0.0384s (+17.2%)
  ➖ test_schema_generation[v1]: 0.0012s → 0.0012s (-0.2%)
  ➖ test_schema_generation[v2]: 0.0020s → 0.0020s (-0.4%)
  ➖ test_simple_json: 0.0004s → 0.0005s (+18.6%)

============================================================
Summary: 1 regressions, 1 improvements, 11 unchanged
============================================================

⚠️  Regressions detected but not failing build (informational only)

Threshold: 1.3x (30% slower triggers a regression warning)

Note: Benchmarks are informational only and won't fail the build.

_{💡 Tip: Download the benchmark-results artifact for detailed JSON data}

codecov · 2026-09-12T08:39:13Z

Codecov Report

❌ Patch coverage is 88.52839% with 99 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/oold/utils/transform.py	80.9%	15 Missing and 14 partials ⚠️
src/oold/utils/labels.py	82.4%	15 Missing and 4 partials ⚠️
src/oold/validation/effective_views.py	86.4%	8 Missing and 5 partials ⚠️
src/oold/utils/readings.py	92.5%	6 Missing and 6 partials ⚠️
src/oold/utils/locator.py	87.5%	8 Missing and 3 partials ⚠️
src/oold/utils/mappings.py	93.8%	3 Missing and 4 partials ⚠️
src/oold/utils/sssom.py	94.6%	3 Missing and 3 partials ⚠️
src/oold/validation/generate.py	93.5%	1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

simontaurus · 2026-09-20T05:01:18Z

Reviewed at 156b9b0 merged with current main (clean merge, 757 pass). The 27 added tests pass, so none of the below is caught by the suite.

Blocking: silent, total data loss. to_rdf never materialises the schema's x-oold-instance-rdf-type as @type, so the frame from_rdf derives matches nothing:

schema = {"@context": CTX, "x-oold-instance-rdf-type": ["ex:Person"], "properties": {...}}
inst   = {"id": "https://ex.org/p1", "name": "Jane", "address": {...}}
transform(inst, [schema], [schema])
# -> {'@context': {...}}        every triple gone, no error

validation.roundtrip.roundtrip() already does this correctly - it sets document["@type"] = instance_rdf_types(schema) when the sample carries none.

The RDF detour is not needed, and the reason is not cost. _apply_rewrites justifies itself with "a single context can only give each term one reading". The implementation being replaced answered that by synthesising one bridge term per synonym and swapping the context under the document - compact under a context where _ex_worksFor means the synonym IRI, redefine that key to the primary IRI, flatten, compact again. It does generalise to x-oold-context: one bridge per synonym, plus one per @type: @vocab property so value positions compact against it.

Measured over the schemas.oo-ld.org cases and synthetic graphs, all three candidates agree on every corpus case and separate only on constructed input:

200 nodes / 800 triples
  as written here (n-quads text round trip) : 133.3 ms
  rewrite pyld's dataset dict in place      :  50.2 ms
  bridge terms, pure JSON-LD                :  60.1 ms

So rewriting the dataset dict is the faster of the two viable options, and it still cannot be made correct. An RDF dataset carries no term definitions, so nothing in it separates a @type: @vocab value - a term drawn from a vocabulary, which a synonym rewrite must follow - from a reference to a node, which it must not touch. A document holding the same IRI in both positions rewrites both or neither. Compaction, expansion and flattening are the normative algorithms; pyld's dataset dict shape is documented nowhere.

Separately, _expand_iri calls jsonld.expand per synonym side: 40 synonyms is 80 processor invocations per from_rdf, where one probe document would do.

Correctness

to_rdf turtle branch (the default) serializes an rdflib Dataset with default_union=False, dropping every named-graph quad; format=NQUADS returns the quad, default returns '\n'.
promote's elif rest: replaces the whole term definition, discarding the primary's @type/@container - a reference exports as a string literal. The else two lines down preserves coercion deliberately.
promote never consults is_exact_match, so a closeMatch tagged with the selected set is promoted and exported as an equivalence - contradicting the module docstring at line 27.
from_rdf frames from schemas[-1] alone, so a base-declared x-oold-instance-rdf-type or embedded property is invisible and a nested node surfaces both embedded and as a sibling. schema_to_frame({"allOf": list(reversed(schemas))}, context) works, since instance_rdf_types and collect_composed_properties both walk allOf.
_apply_rewrites' invert branch adds (obj, pred, subject) without checking obj is not a Literal; rdflib accepts the literal subject and pyld then fails to parse it, one layer later.
null-entry removal is documented but not implemented: synonyms_of drops null entries rather than recording a removal, so a derived schema cannot cancel a base's synonym.
chain's cycle guard keys on id(node), which only holds if the resolver returns the identical object per $ref; a parse-per-call resolver recurses without bound. validation.context_resolution guards with an explicit stack and a depth limit.
_set_ids / is_exact_match / synonyms_of assume dicts and raise AttributeError on a string or list, against the tolerance chain's docstring claims.
_id_keys recognises only the string form of an @id alias, so {"id": {"@id": "@id"}} leaves _:b0 in the output.
Unknown format falls through to turtle, and an unrecognised set_id silently yields the consensus reading, though mapping_sets() knows the valid ones.
The framing decision and the RDF base disagree with validation.roundtrip, which picks framing from embedded_properties(schema) rather than a node count, and threads a base through every pyld call.

Process

Removing the public jsonld_to_jsonld / json_to_json under feat(utils): with no ! cuts a minor release for a breaking removal. Now that main is 1.x this needs feat!: or a BREAKING CHANGE: footer.
The module docstring still says from_rdf takes no set_id; the signature has one and the shape-bridging feature depends on it.
tests/test_transform.py uses banner comments, and neither new module appears in docs/api.md.

github-actions · 2026-09-20T07:24:23Z

📊 Benchmark Results

Click to see benchmark comparison

📊 Benchmark Comparison (threshold: 1.3x)
============================================================

⚠️  Performance Regressions:
  ❌ test_complex_graph: 0.0016s → 0.0032s (+95.2%, ratio: 1.95x)

➖ Unchanged (within threshold):
  ➖ test_simple_dict_document_store: 0.0019s → 0.0020s (+0.7%)
  ➖ test_sqlite_document_store: 0.0021s → 0.0021s (+0.2%)
  ➖ test_local_sparql_store: 0.0379s → 0.0376s (-0.9%)
  ➖ test_oneof_subschema: 0.0598s → 0.0598s (-0.0%)
  ➖ test_enum_docstrings: 0.0503s → 0.0507s (+0.9%)
  ➖ test_subclass_inheritance: 0.0531s → 0.0537s (+1.0%)
  ➖ test_class_hierarchy: 0.0507s → 0.0514s (+1.4%)
  ➖ test_core[v1]: 0.0353s → 0.0357s (+1.2%)
  ➖ test_core[v2]: 0.0428s → 0.0436s (+2.0%)
  ➖ test_schema_generation[v1]: 0.0016s → 0.0016s (+2.0%)
  ➖ test_schema_generation[v2]: 0.0037s → 0.0037s (+0.8%)
  ➖ test_simple_json: 0.0007s → 0.0008s (+17.6%)

============================================================
Summary: 1 regressions, 0 improvements, 12 unchanged
============================================================

⚠️  Regressions detected but not failing build (informational only)

Threshold: 1.3x (30% slower triggers a regression warning)

Note: Benchmarks are informational only and won't fail the build.

_{💡 Tip: Download the benchmark-results artifact for detailed JSON data}

@reverse

- add oold.utils.mappings: promote, mapping_sets, chain, context_of, synonym_entries; the selection OOLD-EXT-4966 describes, complementing validation's promoted_terms which deliberately never chooses - replace the name* alias notation in oold.utils.transform with to_rdf/from_rdf/transform over x-oold-context: exactMatch rewriting, @reverse inversion, @graph framing, @nest shape bridging under a named set (OO-LD/oold-schema#135) - re-encode the alias-notation test cases as x-oold-context, same inputs and expected outputs

@direction

- bridge terms carry the rewrite: compact under a context where the term means the synonym IRI, redefine it to the primary IRI, flatten, recompact - a dataset carries no term definitions, so rewriting RDF cannot tell a @vocab-coerced value from a reference to a node - from_rdf reads a JSON-LD document as it stands instead of sending it through RDF and back, keeping @index and @direction - keep jsonld_to_jsonld and json_to_json, deprecated, with their tests - Dataset.default_union, else triples() drops every quad in a named graph

@reverse

- what a term scopes is a property of the shape, not of the vocabulary - a @reverse synonym lost it, so an embedded object fell back to the consensus terms under a selected set

@context

- inline_scoped resolves a term's scoped @context reference and promotes it with the same mapping set, so an embedded object speaks the vocabulary of the tree it hangs in - ScopeSource states the resolution step as a protocol, keeping the module free of any directory layout - a cyclic reference drops the scoping instead of recursing

@context

- MirrorRule states the IRI-to-path layout as data, so a different tree is a rule rather than a patch - term_chain walks allOf, @context and $ref, which is what a reading needs and wider than the inheritance chain - loader() feeds mappings.chain; a reference the tree cannot reach is skipped, and a cycle terminates

- effective_views promotes each declared set in turn and reports lossless or lossy per reading, which the declared-context checks cannot see - a reading that only changes the document shape is compared as a graph (URDNA2015) rather than as a tree - view_checks reports them as roundtrip.mapping_set records

- readings() returns Turtle plus a Mermaid diagram for the consensus reading and every declared set, and reports a mapping a processor rejects instead of failing the run - blank nodes are skolemised from their URDNA2015 labels, so the same input serialises to the same bytes on every run - named_namespaces names what a serializer would call ns1, ns2

- synth_instance fills a schema from its own examples, defaults and enums only, so a schema with no committed instance still has one to show without inventing a value - a schema whose required properties cannot be filled yields nothing - the inheritance walk it needs is mappings.chain over a Locator

- collect() groups the inline mappings by set, with the consensus ones emitted as a set of their own - crosswalk() chains two sets through the terms they share, and only where one side is an exactMatch - render_set() writes the SSSOM preamble and rows, sorted, so the same mappings give the same file

- a term is labelled by the ontology that owns it, not by one that merely imports it, decided on the local name prefix where there is one - only the IRIs the schemas map are kept, read from the schemas rather than listed - the network lookup is a separate opt-in step, since it is the one part that is neither offline nor reproducible

github-actions · 2026-11-15T11:06:51Z

📊 Benchmark Results

Click to see benchmark comparison

📊 Benchmark Comparison (threshold: 1.3x)
============================================================

⚠️  Performance Regressions:
  ❌ test_complex_graph: 0.0016s → 0.0029s (+86.2%, ratio: 1.86x)

➖ Unchanged (within threshold):
  ➖ test_simple_dict_document_store: 0.0019s → 0.0019s (-0.4%)
  ➖ test_sqlite_document_store: 0.0021s → 0.0021s (+0.2%)
  ➖ test_local_sparql_store: 0.0379s → 0.0392s (+3.3%)
  ➖ test_oneof_subschema: 0.0589s → 0.0599s (+1.7%)
  ➖ test_enum_docstrings: 0.0503s → 0.0501s (-0.5%)
  ➖ test_subclass_inheritance: 0.0530s → 0.0533s (+0.7%)
  ➖ test_class_hierarchy: 0.0506s → 0.0510s (+0.8%)
  ➖ test_core[v1]: 0.0347s → 0.0358s (+3.2%)
  ➖ test_core[v2]: 0.0433s → 0.0437s (+1.0%)
  ➖ test_schema_generation[v1]: 0.0016s → 0.0017s (+0.6%)
  ➖ test_schema_generation[v2]: 0.0037s → 0.0038s (+3.9%)
  ➖ test_simple_json: 0.0007s → 0.0008s (+21.6%)

============================================================
Summary: 1 regressions, 0 improvements, 12 unchanged
============================================================

⚠️  Regressions detected but not failing build (informational only)

Threshold: 1.3x (30% slower triggers a regression warning)

Note: Benchmarks are informational only and won't fail the build.

_{💡 Tip: Download the benchmark-results artifact for detailed JSON data}

simontaurus mentioned this pull request Sep 20, 2026

Drop the local mapping-set selection and transform once oold ships them OO-LD/oold-playgrounds#4

Open

simontaurus and others added 10 commits November 15, 2026 11:30

fix(mappings): keep a promoted term's scoped @context

e87f580

- what a term scopes is a property of the shape, not of the vocabulary - a @reverse synonym lost it, so an embedded object fell back to the consensus terms under a selected set

simontaurus force-pushed the feat/mapping-set-selection branch from d57172b to f1a7b67 Compare November 15, 2026 11:05

Navigation Menu

Repository navigation

Mapping-set selection and rc.2 x-oold-context transform - #154

Mapping-set selection and rc.2 x-oold-context transform#154
simontaurus wants to merge 10 commits into
mainfrom
feat/mapping-set-selection

simontaurus commented Sep 12, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 12, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 12, 2026

Uh oh!

codecov Bot commented Sep 12, 2026 •

edited

Loading

Uh oh!

simontaurus commented Sep 20, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 20, 2026

Uh oh!

github-actions Bot commented Nov 15, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

simontaurus commented Sep 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Notation

Generic tools ported from oold-reference-schemas

Verification

Uh oh!

github-actions Bot commented Sep 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Release preview

Uh oh!

github-actions Bot commented Sep 12, 2026

📊 Benchmark Results

Uh oh!

codecov Bot commented Sep 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

simontaurus commented Sep 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 20, 2026

📊 Benchmark Results

Uh oh!

github-actions Bot commented Nov 15, 2026

📊 Benchmark Results

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

simontaurus commented Sep 12, 2026 •

edited

Loading

github-actions Bot commented Sep 12, 2026 •

edited

Loading

codecov Bot commented Sep 12, 2026 •

edited

Loading

simontaurus commented Sep 20, 2026 •

edited

Loading