Skip to content

[GH-3551] Fix ST_Force3DM and ST_Force4D for XYZM input and mixed-dimension collections - #3552

Open
FurkhanShaikh wants to merge 4 commits into
apache:masterfrom
FurkhanShaikh:fix/gh-3551-force-dimension-handling
Open

FurkhanShaikh wants to merge 4 commits into
apache:masterfrom
FurkhanShaikh:fix/gh-3551-force-dimension-handling

Conversation

@FurkhanShaikh

@FurkhanShaikh FurkhanShaikh commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Did you read the Contributor Guide?

Is this PR related to a ticket?

What changes were proposed in this PR?

Follow-up to #3549, as discussed in its review.

GeometryForce3DMTransformer and GeometryForce4DTransformer decided once per geometry whether Z and M were present, using Functions.hasZ / Functions.hasM, which look only at the first coordinate. That gave wrong results for XYZM input and for geometry collections whose members have different dimensions (examples in #3551).

The decision is now made per coordinate sequence, in a small package-private base class GeometryForceTransformer that both transformers extend. Ordinates of the target layout that the source already has are kept; missing or NaN Z/M ordinates are filled with the supplied defaults.

As requested in review, the base class also keeps geometry types and empty members. JTS GeometryTransformer turns a MULTI* with one member into that member (it rebuilds them with GeometryFactory.buildGeometry), and it drops empty members and empty holes. The base class converts one member at a time instead, at any depth, using the input's factory, so the result keeps the input's SRID even when members carry another one. Empty members are converted to the target layout too, since a multipart geometry needs one layout for all of its parts to be serialized.

Behavior changes:

  • ST_Force3DM drops Z from XYZM input instead of returning it unchanged (POINT ZM(1 2 3 4) -> POINT M(1 2 4)).
  • ST_Force3DM keeps existing M values in mixed-dimension collections instead of overwriting them with the default.
  • ST_Force4D keeps existing Z and M values in mixed-dimension collections and no longer writes NaN Z or M.
  • NaN Z/M ordinates in an input that already has the target layout are filled with the default, consistent with how ST_Force3D already treats NaN Z.
  • A single-member MULTI* that is converted keeps its type (ST_Force4D(MULTIPOINT((1 2)), 5, 7) -> MULTIPOINT ZM((1 2 5 7)); master returned POINT ZM(1 2 5 7)), at the top level and nested in collections.
  • Empty members and empty holes of a converted geometry are kept, in the target layout (ST_Force3DM(MULTIPOINT((1 2), EMPTY), 9) -> MULTIPOINT M((1 2 9), EMPTY); master returned POINT M(1 2 9)).
  • A polygon with a ring of three points that is converted stays a polygon (ST_Force4D(POLYGON((0 0, 0 4, 0 0)), 5, 7) -> POLYGON ZM((0 0 5 7, 0 4 5 7, 0 0 5 7)); master returned a LINESTRING).

For converted geometries, the single-member MULTI* and empty-member changes match PostGIS. A geometry that is empty as a whole is still returned unchanged. ST_Force2D and ST_Force3D are not changed.

How was this patch tested?

  • New assertions in FunctionsTest.testForce3DM for XYZM input, XYZ input and a mixed-dimension collection.
  • New mixed-dimension collection case in FunctionsTest.force4D.
  • New tests force3DMKeepsMultiTypes, force3DMKeepsEmptyMembers, force4DKeepsMultiTypes and force4DKeepsEmptyMembers: single-member MULTI* at the top level and nested, polygons with holes and with empty holes, empty members in collections and in each multi type, NaN Z and M ordinates, a multi point assembled from serialized parts with different layouts as ST_Collect can produce it, and inputs already in the target layout (returned as the same object). Apart from the same-object checks, input and result go through GeometrySerializer, as in Spark, and the layout of every part is checked.
  • New tests forceDimensionKeepsInputSrid, forceDimensionRejectsUnknownSubtypes and forceDimensionKeepsThreePointRings.
  • New cases in the ST_Force3DM and ST_Force4D Spark SQL tests in functionTestScala: XYZM input, a single-member MULTI*, multis nested in a collection, and empty members.
  • Existing FunctionsTest (including force4DIsThreadSafe from [GH-3548] Fix ST_Force4D thread safety #3549), PredicatesTest and CoverageValidationTest pass.

Did this PR include necessary documentation updates?

  • No, this PR does not affect any public API so no need to change the documentation.

…ed-dimension collections

Both transformers decided once per geometry, from the first coordinate,
whether Z and M were present. The decision is now made per coordinate
sequence in a small shared base class: ordinates the source already has
are kept, and missing or NaN Z/M ordinates are filled with the defaults.

- ST_Force3DM now drops Z from XYZM input instead of returning it unchanged.
- ST_Force3DM keeps existing M values in mixed-dimension collections.
- ST_Force4D keeps existing Z/M values in mixed-dimension collections and
  no longer writes NaN Z.

Generated-by: AI coding assistants (Gemini, GPT, Claude)
@FurkhanShaikh

Copy link
Copy Markdown
Contributor Author

While testing further I found a side effect of this PR that I should flag. Because inputs are now checked per coordinate sequence, some geometries that master returned untouched now go through GeometryTransformer, which collapses a single-member MULTI* to its member. Examples:

  • ST_Force4D(MULTILINESTRING ZM((0 0 1 2, 1 1 NaN 2)), 5, 7): master returns the input unchanged; this PR returns LINESTRING ZM(0 0 1 2, 1 1 5 2).
  • ST_Force3DM(MULTIPOINT ZM((1 2 3 4)), 7): master returns the input unchanged; this PR returns POINT M(1 2 4).

I have a small addition ready (about 25 lines plus a test) that re-wraps the result so MULTI* types are kept in ST_Force3DM and ST_Force4D, the same way GH-2525 did for ST_Force3D on MULTIPOLYGON. It also stops the existing collapse for 2D inputs to these two functions; empty-member handling is unchanged.

I know you wanted geometry-type preservation reviewed separately. Would you prefer I add it to this PR so it doesn't introduce new collapses, or keep this PR as is and send the type change as a follow-up?


/** Returns the input unchanged when it is empty or already entirely in the target layout. */
protected final Geometry transformIfNeeded(Geometry geometry) {
return geometry.isEmpty() || hasTargetCoordinates(geometry) ? geometry : transform(geometry);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's include the preservation fix here, including nested MULTI* members.

Empty members also regress. I tested ST_Force4D(ST_GeomFromText('GEOMETRYCOLLECTION(POINT ZM(1 2 3 4), LINESTRING EMPTY)'), 5, 7): the base keeps both members, but this PR drops the empty line.

Please add tests for nested multis and empty members in both functions. Re-wrapping just the top-level result won't cover these cases.

@FurkhanShaikh FurkhanShaikh Nov 17, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, done in 16c35b5, 680a152 and fa416bf.

Instead of re-wrapping the result, the transformer now overrides the collection and polygon methods of GeometryTransformer and converts one member at a time. MULTI* types are kept at any depth, and empty members and empty holes are no longer dropped.

Empty members of a converted geometry are converted to the target layout along with the rest, as PostGIS does, because a multipart geometry has to have one layout for all of its parts to be serialized. Your example ST_Force4D(ST_GeomFromText('GEOMETRYCOLLECTION(POINT ZM(1 2 3 4), LINESTRING EMPTY)'), 5, 7) now returns GEOMETRYCOLLECTION ZM(POINT ZM(1 2 3 4), LINESTRING ZM EMPTY), with both members. A geometry that is empty as a whole is still returned unchanged, as before.

New tests in FunctionsTest, for both functions: force3DMKeepsMultiTypes, force3DMKeepsEmptyMembers, force4DKeepsMultiTypes and force4DKeepsEmptyMembers. They cover single-member MULTI* at the top level and nested two levels deep, polygons with holes and with empty holes, empty members in collections and in each multi type, NaN Z and M ordinates, and a multi point assembled from serialized parts with different layouts, the way ST_Collect can produce it. Apart from the checks that an input already in the target layout is returned as the same object, their input goes through the geometry serializer and the result is serialized again, as in Spark, and the layout of every part, empty ones included, is checked.

Three more tests came out of this: forceDimensionKeepsInputSrid (the result keeps the input's SRID when members carry another one, as ST_Collect can produce), forceDimensionRejectsUnknownSubtypes, and forceDimensionKeepsThreePointRings (a polygon whose ring has three points stays a polygon; GeometryTransformer turns such a ring into a LineString, so master returned a LINESTRING).

The Spark SQL tests in functionTestScala also have new cases for both functions: a single-member MULTI*, multis nested in a collection, a collection with an empty member, and a multi point with an empty member. Their inputs are XY throughout: on Spark 4.2 the native geometry type rejects a collection whose members have different layouts, such as your example, before the function runs (that failed the 4.2 build on 680a152 and is fixed in fa416bf). Your example itself is covered in FunctionsTest.

Compared with master, for inputs these functions convert: a single-member MULTI* now keeps its type (master returned the member), and empty members and empty holes are kept (master dropped them). I've added both to the PR description.

…and ST_Force4D

GeometryTransformer turns a MULTI* with one member into that member
(it rebuilds them with GeometryFactory.buildGeometry), and it drops
empty members and empty holes. Convert one member at a time instead,
so the type and every member of the input are kept at any depth.

Empty members are converted to the target layout as well, since a
multipart geometry needs one layout for all of its parts to be
serialized. Members are converted with the input's factory, so the
result keeps the input's SRID and precision model when members carry
other ones. A non-empty geometry of a subtype other than the JTS ones
is rejected with an IllegalArgumentException instead of recursing
endlessly.

The tests run the functions on serialized input and serialize the
result, as Spark does, and check the layout of every part.

Generated-by: AI coding assistants (Claude)
…nd ST_Force4D

GeometryTransformer.transformLinearRing returns a LineString for a ring
of fewer than four points, which a closed ring of three points is. The
polygon override cast that result to LinearRing, so such a polygon, on
its own, as a hole or inside a multi polygon or collection, failed with
a ClassCastException. Rebuild the ring from its converted coordinates
instead, so the polygon keeps its type.

Also add Spark SQL cases for single-member MULTI* inputs, nested multis
and empty members to the ST_Force3DM and ST_Force4D tests.

Generated-by: AI coding assistants (Claude)
…rce4D Spark tests

On Spark 4.2 a geometry is passed between expressions as a native
Spark geometry, which does not accept a collection whose members have
different coordinate layouts. Two of the new Spark SQL cases used such
a collection as input and failed there with WKB_PARSE_ERROR before
ST_Force3DM or ST_Force4D ran. Use XY collections instead; the
mixed-layout cases stay in FunctionsTest.

Generated-by: AI coding assistants (Claude)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ST_Force3DM and ST_Force4D mishandle XYZM input and geometry collections with mixed dimensions

2 participants