From ff581cc5881e72d6248fc7bab5ff02fd4cbabee1 Mon Sep 17 00:00:00 2001 From: Jerry Combs Date: Wed, 18 Nov 2026 07:35:08 -0600 Subject: [PATCH] import-ldif: a second Library inside the driver set, links by DN (#133) A DirXML-Library under the driver set other than cn=Library (cn=SKyPRO) was read as a driver: its objects went into library/ as the set's own, and links to them pointed at drivers/SKyPRO/..., which resolve to nothing. Its objects, and those in any container that is no driver or channel, are now read-only references that keep their DN; a link finds its target by the DN it names; an unresolved link is listed with that DN. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 9 +++++ docs/commands.md | 2 +- docs/tree-layout.md | 6 ++- .../java/com/pointblue/dirxml/dev/Cli.java | 2 +- .../dirxml/dev/model/PolicyLink.java | 2 + .../dirxml/dev/source/LdifReader.java | 33 ++++++++++++---- .../dirxml/dev/source/LdifReaderTest.java | 38 ++++++++++++++++--- 7 files changed, 75 insertions(+), 17 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 740154b0..6b132a12 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,15 @@ and this project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html) ## [Unreleased] +### Fixed + +- **`import-ldif` with a second Library inside the driver set (#133):** a DirXML-Library under the driver set + other than `cn=Library` (`cn=SKyPRO,`) was read as a driver. Its objects landed in `library/` + as the driver set's own, and the links to them pointed at `drivers/SKyPRO/…`, which resolve to nothing. Its + objects, and those in any container that is no driver or channel, are now read-only references that keep + their DN, like other Libraries' objects. A link now finds its target by the DN it names, so the two always + agree. A link that resolves to nothing is listed with that DN. + ## [0.73.0] - 2026-11-16 ### Fixed diff --git a/docs/commands.md b/docs/commands.md index e6c232c3..b3933543 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -135,7 +135,7 @@ More: [getting-started.md](getting-started.md#4-import-the-driver-set). Reads an LDIF of the driver-set subtree and writes an IDM-as-code tree. The LDIF is a subtree export from the driver set's DN down, with `objectClass` and the DirXML data attributes. Same writing and exit behaviour as `import`. -An LDIF of more than that (a whole vault, with several driver sets and Libraries outside them) works too, with `--driver-set` naming the driver set. The tree takes that driver set's own drivers and objects, found by DN: another driver set's driver or Library object of the same name stays out. From the other Libraries it takes every object of a Library outside every driver set, and what the driver set links from another driver set's Library, as read-only references ([tree-layout.md](tree-layout.md#library)). When two of these have one name, the tree keeps the driver set's own and a `NOTE:` line names the other. +An LDIF of more than that (a whole vault, with several driver sets and Libraries outside them) works too, with `--driver-set` naming the driver set. The tree takes that driver set's own drivers and objects, found by DN: another driver set's driver or Library object of the same name stays out. From the other Libraries it takes every object of a Library outside every driver set, and what the driver set links from another driver set's Library, as read-only references ([tree-layout.md](tree-layout.md#library)), as it does for a second Library inside the driver set. A link finds its target by the DN it names; a link that resolves to nothing is listed with that DN. When two of these have one name, the tree keeps the driver set's own and a `NOTE:` line names the other. ``` bin/idm import-ldif [--driver-set DN] diff --git a/docs/tree-layout.md b/docs/tree-layout.md index e539ce4c..335ebc50 100644 --- a/docs/tree-layout.md +++ b/docs/tree-layout.md @@ -128,8 +128,10 @@ the file stays in `library/`. A driver set can also use Libraries in other containers of the vault. A Designer project records them, and `import-project` puts their objects in `library/` too, each with a `library.dn` entry in `library.xml` naming the Library it lives -in. `import-ldif`, given an LDIF of more than the driver set, does the same for -every object of a Library outside every driver set, and for the objects the +in. `import-ldif` does the same for a second Library inside the driver set (any +Library but `cn=Library,`) and for a policy in a container that is +no driver or channel. Given an LDIF of more than the driver set, it also does it +for every object of a Library outside every driver set, and for the objects the driver set links from another driver set's Library. These are read-only references: - **Links** to them resolve, and a deploy writes their real DN into a driver's links. diff --git a/src/main/java/com/pointblue/dirxml/dev/Cli.java b/src/main/java/com/pointblue/dirxml/dev/Cli.java index 4732f91a..b8cd2dbc 100644 --- a/src/main/java/com/pointblue/dirxml/dev/Cli.java +++ b/src/main/java/com/pointblue/dirxml/dev/Cli.java @@ -522,7 +522,7 @@ private static void report(DriverSet ds) { + "referenced policies, or use a driver-set export/LDIF/live), or a dangling reference " + "in the source itself:"); for (PolicyLink l : broken) { - System.out.println(" - " + l); + System.out.println(" - " + l + (l.sourceDn == null ? "" : " (" + l.sourceDn + ")")); } } } diff --git a/src/main/java/com/pointblue/dirxml/dev/model/PolicyLink.java b/src/main/java/com/pointblue/dirxml/dev/model/PolicyLink.java index d05cf98a..052e2429 100644 --- a/src/main/java/com/pointblue/dirxml/dev/model/PolicyLink.java +++ b/src/main/java/com/pointblue/dirxml/dev/model/PolicyLink.java @@ -11,6 +11,8 @@ public final class PolicyLink { public final PolicySet set; public String ref; // artifact path, e.g. "drivers/AD/subscriber/sub-etp_Scoping" public int order; + /** The DN the link named, when it was read from a directory (LDIF or live); for messages only, never written. */ + public String sourceDn; public PolicyLink(PolicySet set, String ref, int order) { this.set = Objects.requireNonNull(set, "set"); diff --git a/src/main/java/com/pointblue/dirxml/dev/source/LdifReader.java b/src/main/java/com/pointblue/dirxml/dev/source/LdifReader.java index 977f8d41..ed325f42 100644 --- a/src/main/java/com/pointblue/dirxml/dev/source/LdifReader.java +++ b/src/main/java/com/pointblue/dirxml/dev/source/LdifReader.java @@ -186,7 +186,8 @@ public static DriverSet fromEntries(Collection entries, String sourceName } } - // 3. artifacts, placed by DN structure + // 3. artifacts, placed by DN structure; each one's path by its DN, so a link finds it wherever it is + Map refByDn = new HashMap<>(); for (Entry e : entries) { boolean policy = e.hasClass("DirXML-Rule") || e.hasClass("DirXML-StyleSheet"); // a package's GCV definitions: class DirXML-GlobalConfigDef, content in @@ -204,12 +205,20 @@ public static DriverSet fromEntries(Collection entries, String sourceName continue; } } - Placement p = otherLibrary != null ? new Placement(Scope.LIBRARY, null) : place(e.dn, dsDn); + String parent = parentDn(e.dn); + Placement p = otherLibrary != null || libraries.contains(norm(parent)) + ? new Placement(Scope.LIBRARY, null) : place(e.dn, dsDn); Driver d = p.driver == null ? null : driversByLowerName.get(p.driver.toLowerCase()); if (p.scope != Scope.LIBRARY && d == null) { - // an artifact under an unknown driver — keep it in the library, flagged + // in a container that is no driver of the set's: kept in the library p = new Placement(Scope.LIBRARY, null); } + if (otherLibrary == null && p.scope == Scope.LIBRARY && !sameDn(parent, dsDn) + && !sameDn(parent, "cn=Library," + dsDn)) { + // another Library inside the set (#133: cn=SKyPRO,), or another container: the tree's + // Library is cn=Library,, so this one is a read-only reference that keeps its DN + otherLibrary = parent; + } String name = rdn(e.dn); if (policy) { Policy pol = new Policy(name, p.scope, p.driver, xmlOrNull(e.first("XmlData"))); @@ -218,6 +227,7 @@ public static DriverSet fromEntries(Collection entries, String sourceName pol.meta.put(DuplicateArtifacts.LIBRARY_DN, otherLibrary); } copyMeta(e, pol.meta, e.hasClass("DirXML-StyleSheet") ? "DirXML-StyleSheet" : "DirXML-Rule"); + refByDn.put(norm(e.dn), pol.path()); if (p.scope == Scope.LIBRARY) { ds.library.policies.add(pol); } else if (p.scope == Scope.SUBSCRIBER) { @@ -249,6 +259,7 @@ public static DriverSet fromEntries(Collection entries, String sourceName r.meta.put(DuplicateArtifacts.LIBRARY_DN, otherLibrary); } copyMeta(e, r.meta, gcvDef ? "DirXML-GlobalConfigDef" : "DirXML-Resource"); + refByDn.put(norm(e.dn), r.path()); if (p.scope == Scope.LIBRARY) { ds.library.resources.add(r); } else { @@ -394,11 +405,17 @@ public static DriverSet fromEntries(Collection entries, String sourceName d.meta.put("linkage.unknown." + (n++), link); continue; } - // a target outside the set is in another Library: the tree keeps it in library/ - Placement p = isUnder(policyDn, dsDn) ? place(policyDn, dsDn) : new Placement(Scope.LIBRARY, null); - String ref = com.pointblue.dirxml.dev.model.Artifact.path( - p.scope == Scope.LIBRARY ? Scope.LIBRARY : p.scope, p.driver, rdn(policyDn)); - d.links.add(new PolicyLink(set, ref, order)); + // the target's path as read, by its DN; a target the entries don't hold, by the DN's shape + // (outside the set: another Library, which the tree keeps in library/) + String ref = refByDn.get(norm(policyDn)); + if (ref == null) { + Placement p = isUnder(policyDn, dsDn) ? place(policyDn, dsDn) : new Placement(Scope.LIBRARY, null); + ref = com.pointblue.dirxml.dev.model.Artifact.path( + p.scope == Scope.LIBRARY ? Scope.LIBRARY : p.scope, p.driver, rdn(policyDn)); + } + PolicyLink pl = new PolicyLink(set, ref, order); + pl.sourceDn = policyDn; + d.links.add(pl); } } diff --git a/src/test/java/com/pointblue/dirxml/dev/source/LdifReaderTest.java b/src/test/java/com/pointblue/dirxml/dev/source/LdifReaderTest.java index 4d52f8e1..6c93e636 100644 --- a/src/test/java/com/pointblue/dirxml/dev/source/LdifReaderTest.java +++ b/src/test/java/com/pointblue/dirxml/dev/source/LdifReaderTest.java @@ -242,12 +242,21 @@ private static List vault() { entry("cn=shared-only," + global, "DirXML-Rule", "XmlData", String.format(rule, "shared only")), entry("cn=other-linked,cn=Library," + ds2, "DirXML-Rule", "XmlData", String.format(rule, "other, linked")), entry("cn=other-unlinked,cn=Library," + ds2, "DirXML-Rule", "XmlData", String.format(rule, "other, not linked")), + // a second Library inside the set (klasen's cn=SKyPRO), and a policy in a container that is no driver + entry("cn=SKyPRO," + ds1, "DirXML-Library"), + entry("cn=SPACDMCS-EcmaScript,cn=SKyPRO," + ds1, "DirXML-Resource", + "DirXML-ContentType", "text/ecmascript", "DirXML-Data", "function f(){}"), + entry("cn=pub-ets-ps-Relationships,cn=SKyPRO," + ds1, "DirXML-Rule", "XmlData", String.format(rule, "second Library")), + entry("cn=pub,cn=AD," + ds1, "organizationalUnit"), + entry("cn=pub-etp-ValidityPeriodLog,cn=pub,cn=AD," + ds1, "DirXML-Rule", "XmlData", String.format(rule, "odd container")), // a driver of one name in each set, each with a publisher policy of one name entry("cn=AD," + ds1, "DirXML-Driver", "DirXML-Policies", "cn=pub-etp,cn=Publisher,cn=AD," + ds1 + "#0#2", "DirXML-Policies", "cn=pub-ets-ps-Relationships,cn=Library," + ds1 + "#1#2", "DirXML-Policies", "cn=shared-only," + global + "#2#2", - "DirXML-Policies", "cn=other-linked,cn=Library," + ds2 + "#3#2"), + "DirXML-Policies", "cn=other-linked,cn=Library," + ds2 + "#3#2", + "DirXML-Policies", "cn=SPACDMCS-EcmaScript,cn=SKyPRO," + ds1 + "#3#11", + "DirXML-Policies", "cn=pub-etp-ValidityPeriodLog,cn=pub,cn=AD," + ds1 + "#4#2"), entry("cn=Publisher,cn=AD," + ds1, "DirXML-Publisher"), entry("cn=pub-etp,cn=Publisher,cn=AD," + ds1, "DirXML-Rule", "XmlData", String.format(rule, "own AD")), entry("cn=Entitlement1,cn=AD," + ds1, "DirXML-Entitlement", "XmlData", ""), @@ -298,7 +307,10 @@ public void rdnsAreUniqueOnlyInTheirContainer() { assertTrue(CanonicalXmlText.of(own).contains("own")); assertNull(own.meta.get(com.pointblue.dirxml.dev.model.DuplicateArtifacts.LIBRARY_DN)); assertEquals(1, ds.duplicates.size()); - assertTrue(ds.duplicates.get(0).note, ds.duplicates.get(0).note.contains("in cn=Library,o=services")); + String note = ds.duplicates.get(0).note; + assertTrue(note, note.contains("3 objects are named library/pub-ets-ps-Relationships")); + assertTrue(note, note.contains("in cn=Library,o=services") && note.contains("in cn=SKyPRO,cn=DriverSet-Identity")); + assertTrue(ds.duplicates.get(0).advice, ds.duplicates.get(0).advice.startsWith("The tree has one Library")); Policy shared = (Policy) ds.resolve("library/shared-only"); assertEquals("cn=Library,o=services", shared.meta.get(com.pointblue.dirxml.dev.model.DuplicateArtifacts.LIBRARY_DN)); @@ -307,11 +319,27 @@ public void rdnsAreUniqueOnlyInTheirContainer() { assertEquals("cn=Library,cn=DriverSet-Other,ou=services,o=identity", linked.meta.get(com.pointblue.dirxml.dev.model.DuplicateArtifacts.LIBRARY_DN)); assertNull(ds.resolve("library/other-unlinked")); - assertEquals(3, ds.library.policies.size()); - // every link resolves, the other Libraries' targets to library/ + // the second Library inside the set, and the odd container: in library/, read-only, keeping their DNs + Resource script = (Resource) ds.resolve("library/SPACDMCS-EcmaScript"); + assertEquals("cn=SKyPRO,cn=DriverSet-Identity,ou=services,o=identity", + script.meta.get(com.pointblue.dirxml.dev.model.DuplicateArtifacts.LIBRARY_DN)); + Policy odd = (Policy) ds.resolve("library/pub-etp-ValidityPeriodLog"); + assertEquals("cn=pub,cn=AD,cn=DriverSet-Identity,ou=services,o=identity", + odd.meta.get(com.pointblue.dirxml.dev.model.DuplicateArtifacts.LIBRARY_DN)); + assertEquals(4, ds.library.policies.size()); + assertEquals(1, ds.library.resources.size()); + + // every link resolves (#133: drivers/SKyPRO/SPACDMCS-EcmaScript did not), by the DN it names assertTrue(ds.unresolvedLinks().toString(), ds.unresolvedLinks().isEmpty()); - assertEquals(4, ad.links.size()); + assertEquals(6, ad.links.size()); + PolicyLink toScript = ad.links.stream().filter(l -> l.ref.equals("library/SPACDMCS-EcmaScript")).findFirst().orElseThrow(); + assertEquals("cn=SPACDMCS-EcmaScript,cn=SKyPRO,cn=DriverSet-Identity,ou=services,o=identity", toScript.sourceDn); + // and a deploy writes each link with the DN it was read from + List values = com.pointblue.dirxml.dev.deploy.VaultMapping.policiesValues(ds.dn, ad, ds); + assertTrue(values.toString(), values.contains("cn=SPACDMCS-EcmaScript,cn=SKyPRO,cn=DriverSet-Identity,ou=services,o=identity#3#11")); + assertTrue(values.toString(), values.contains("cn=pub-etp-ValidityPeriodLog,cn=pub,cn=AD,cn=DriverSet-Identity,ou=services,o=identity#4#2")); + assertTrue(values.toString(), values.contains("cn=shared-only,cn=Library,o=services#2#2")); // and the other set reads the same way, with its own AD DriverSet other = LdifReader.fromEntries(vault(), "vault.ldif", Map.of(), "cn=DriverSet-Other,ou=services,o=identity");