Repository navigation
[GH-3551] Fix ST_Force3DM and ST_Force4D for XYZM input and mixed-dimension collections - #3552
FurkhanShaikh wants to merge 4 commits into
Conversation
…ed-dimension collections Both transformers decided once per geometry, from the first coordinate, whether Z and M were present. The decision is now made per coordinate sequence in a small shared base class: ordinates the source already has are kept, and missing or NaN Z/M ordinates are filled with the defaults. - ST_Force3DM now drops Z from XYZM input instead of returning it unchanged. - ST_Force3DM keeps existing M values in mixed-dimension collections. - ST_Force4D keeps existing Z/M values in mixed-dimension collections and no longer writes NaN Z. Generated-by: AI coding assistants (Gemini, GPT, Claude)
|
While testing further I found a side effect of this PR that I should flag. Because inputs are now checked per coordinate sequence, some geometries that master returned untouched now go through
I have a small addition ready (about 25 lines plus a test) that re-wraps the result so I know you wanted geometry-type preservation reviewed separately. Would you prefer I add it to this PR so it doesn't introduce new collapses, or keep this PR as is and send the type change as a follow-up? |
|
|
||
| /** Returns the input unchanged when it is empty or already entirely in the target layout. */ | ||
| protected final Geometry transformIfNeeded(Geometry geometry) { | ||
| return geometry.isEmpty() || hasTargetCoordinates(geometry) ? geometry : transform(geometry); |
There was a problem hiding this comment.
Let's include the preservation fix here, including nested MULTI* members.
Empty members also regress. I tested ST_Force4D(ST_GeomFromText('GEOMETRYCOLLECTION(POINT ZM(1 2 3 4), LINESTRING EMPTY)'), 5, 7): the base keeps both members, but this PR drops the empty line.
Please add tests for nested multis and empty members in both functions. Re-wrapping just the top-level result won't cover these cases.
There was a problem hiding this comment.
Thanks, done in 16c35b5, 680a152 and fa416bf.
Instead of re-wrapping the result, the transformer now overrides the collection and polygon methods of GeometryTransformer and converts one member at a time. MULTI* types are kept at any depth, and empty members and empty holes are no longer dropped.
Empty members of a converted geometry are converted to the target layout along with the rest, as PostGIS does, because a multipart geometry has to have one layout for all of its parts to be serialized. Your example ST_Force4D(ST_GeomFromText('GEOMETRYCOLLECTION(POINT ZM(1 2 3 4), LINESTRING EMPTY)'), 5, 7) now returns GEOMETRYCOLLECTION ZM(POINT ZM(1 2 3 4), LINESTRING ZM EMPTY), with both members. A geometry that is empty as a whole is still returned unchanged, as before.
New tests in FunctionsTest, for both functions: force3DMKeepsMultiTypes, force3DMKeepsEmptyMembers, force4DKeepsMultiTypes and force4DKeepsEmptyMembers. They cover single-member MULTI* at the top level and nested two levels deep, polygons with holes and with empty holes, empty members in collections and in each multi type, NaN Z and M ordinates, and a multi point assembled from serialized parts with different layouts, the way ST_Collect can produce it. Apart from the checks that an input already in the target layout is returned as the same object, their input goes through the geometry serializer and the result is serialized again, as in Spark, and the layout of every part, empty ones included, is checked.
Three more tests came out of this: forceDimensionKeepsInputSrid (the result keeps the input's SRID when members carry another one, as ST_Collect can produce), forceDimensionRejectsUnknownSubtypes, and forceDimensionKeepsThreePointRings (a polygon whose ring has three points stays a polygon; GeometryTransformer turns such a ring into a LineString, so master returned a LINESTRING).
The Spark SQL tests in functionTestScala also have new cases for both functions: a single-member MULTI*, multis nested in a collection, a collection with an empty member, and a multi point with an empty member. Their inputs are XY throughout: on Spark 4.2 the native geometry type rejects a collection whose members have different layouts, such as your example, before the function runs (that failed the 4.2 build on 680a152 and is fixed in fa416bf). Your example itself is covered in FunctionsTest.
Compared with master, for inputs these functions convert: a single-member MULTI* now keeps its type (master returned the member), and empty members and empty holes are kept (master dropped them). I've added both to the PR description.
…and ST_Force4D GeometryTransformer turns a MULTI* with one member into that member (it rebuilds them with GeometryFactory.buildGeometry), and it drops empty members and empty holes. Convert one member at a time instead, so the type and every member of the input are kept at any depth. Empty members are converted to the target layout as well, since a multipart geometry needs one layout for all of its parts to be serialized. Members are converted with the input's factory, so the result keeps the input's SRID and precision model when members carry other ones. A non-empty geometry of a subtype other than the JTS ones is rejected with an IllegalArgumentException instead of recursing endlessly. The tests run the functions on serialized input and serialize the result, as Spark does, and check the layout of every part. Generated-by: AI coding assistants (Claude)
…nd ST_Force4D GeometryTransformer.transformLinearRing returns a LineString for a ring of fewer than four points, which a closed ring of three points is. The polygon override cast that result to LinearRing, so such a polygon, on its own, as a hole or inside a multi polygon or collection, failed with a ClassCastException. Rebuild the ring from its converted coordinates instead, so the polygon keeps its type. Also add Spark SQL cases for single-member MULTI* inputs, nested multis and empty members to the ST_Force3DM and ST_Force4D tests. Generated-by: AI coding assistants (Claude)
…rce4D Spark tests On Spark 4.2 a geometry is passed between expressions as a native Spark geometry, which does not accept a collection whose members have different coordinate layouts. Two of the new Spark SQL cases used such a collection as input and failed there with WKB_PARSE_ERROR before ST_Force3DM or ST_Force4D ran. Use XY collections instead; the mixed-layout cases stay in FunctionsTest. Generated-by: AI coding assistants (Claude)
Did you read the Contributor Guide?
Is this PR related to a ticket?
[GH-XXX] my subject. Closes ST_Force3DM and ST_Force4D mishandle XYZM input and geometry collections with mixed dimensions #3551What changes were proposed in this PR?
Follow-up to #3549, as discussed in its review.
GeometryForce3DMTransformerandGeometryForce4DTransformerdecided once per geometry whether Z and M were present, usingFunctions.hasZ/Functions.hasM, which look only at the first coordinate. That gave wrong results for XYZM input and for geometry collections whose members have different dimensions (examples in #3551).The decision is now made per coordinate sequence, in a small package-private base class
GeometryForceTransformerthat both transformers extend. Ordinates of the target layout that the source already has are kept; missing orNaNZ/M ordinates are filled with the supplied defaults.As requested in review, the base class also keeps geometry types and empty members. JTS
GeometryTransformerturns aMULTI*with one member into that member (it rebuilds them withGeometryFactory.buildGeometry), and it drops empty members and empty holes. The base class converts one member at a time instead, at any depth, using the input's factory, so the result keeps the input's SRID even when members carry another one. Empty members are converted to the target layout too, since a multipart geometry needs one layout for all of its parts to be serialized.Behavior changes:
ST_Force3DMdrops Z from XYZM input instead of returning it unchanged (POINT ZM(1 2 3 4)->POINT M(1 2 4)).ST_Force3DMkeeps existing M values in mixed-dimension collections instead of overwriting them with the default.ST_Force4Dkeeps existing Z and M values in mixed-dimension collections and no longer writesNaNZ or M.NaNZ/M ordinates in an input that already has the target layout are filled with the default, consistent with howST_Force3Dalready treatsNaNZ.MULTI*that is converted keeps its type (ST_Force4D(MULTIPOINT((1 2)), 5, 7)->MULTIPOINT ZM((1 2 5 7)); master returnedPOINT ZM(1 2 5 7)), at the top level and nested in collections.ST_Force3DM(MULTIPOINT((1 2), EMPTY), 9)->MULTIPOINT M((1 2 9), EMPTY); master returnedPOINT M(1 2 9)).ST_Force4D(POLYGON((0 0, 0 4, 0 0)), 5, 7)->POLYGON ZM((0 0 5 7, 0 4 5 7, 0 0 5 7)); master returned aLINESTRING).For converted geometries, the single-member
MULTI*and empty-member changes match PostGIS. A geometry that is empty as a whole is still returned unchanged.ST_Force2DandST_Force3Dare not changed.How was this patch tested?
FunctionsTest.testForce3DMfor XYZM input, XYZ input and a mixed-dimension collection.FunctionsTest.force4D.force3DMKeepsMultiTypes,force3DMKeepsEmptyMembers,force4DKeepsMultiTypesandforce4DKeepsEmptyMembers: single-memberMULTI*at the top level and nested, polygons with holes and with empty holes, empty members in collections and in each multi type,NaNZ and M ordinates, a multi point assembled from serialized parts with different layouts asST_Collectcan produce it, and inputs already in the target layout (returned as the same object). Apart from the same-object checks, input and result go throughGeometrySerializer, as in Spark, and the layout of every part is checked.forceDimensionKeepsInputSrid,forceDimensionRejectsUnknownSubtypesandforceDimensionKeepsThreePointRings.ST_Force3DMandST_Force4DSpark SQL tests infunctionTestScala: XYZM input, a single-memberMULTI*, multis nested in a collection, and empty members.FunctionsTest(includingforce4DIsThreadSafefrom [GH-3548] Fix ST_Force4D thread safety #3549),PredicatesTestandCoverageValidationTestpass.Did this PR include necessary documentation updates?