Skip to content

[GH-3178] Add a k-nearest-neighbour index for Geography - #3543

Merged
jiayuasu merged 6 commits into
apache:masterfrom
jiayuasu:feature/geography-knn-index
Nov 17, 2026
Merged

jiayuasu merged 6 commits into
apache:masterfrom
jiayuasu:feature/geography-knn-index

Conversation

@jiayuasu

@jiayuasu jiayuasu commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

Did you read the Contributor Guide?

Is this PR related to a ticket?

What changes were proposed in this PR?

This adds org.apache.sedona.common.geography.GeographyKnnIndex<T>, an in-memory index that answers k-nearest-neighbour queries between geographies. Neighbours are ranked by Functions.distance(Geography, Geography), the minimum distance on the sphere in meters, which is what ST_Distance returns for two geographies.

It is the search structure behind the geography ST_KNN join in #3542. It is split out and placed in the common module, with a generic item type, so that the broadcast join, the partitioned join that follows it, and other engines can share it. Nothing calls it yet in this PR.

  • Each geography is stored by its latitude / longitude bounding rectangle in an STRtree. The tree is only used as a hierarchy of rectangles.
  • nearest(query, k, includeTies) runs a best-first search over the tree. Nodes are ordered and pruned by the distance on the sphere between the query rectangle and the node rectangle (S2LatLngRect.getDistance), which is a lower bound of the distance to anything under the node. A planar distance in degrees between the same rectangles is not a lower bound across the antimeridian or near the poles.
  • merge(a, b, k, includeTies) combines the neighbours that two disjoint sets of geographies produced for the same query, with the same tie rule as nearest.
  • bounds(Geography) and toEnvelope(S2LatLngRect) expose the bounding rectangle of a geography and the planar envelope that places it in a planar index. A rectangle that crosses the antimeridian gets the full longitude range.
  • distanceLowerBound(queryRect, envelope) exposes the pruning bound in meters, so that a nearest-neighbour search over another tree of such envelopes can use the same bound.
  • The bounding rectangle is Geography.region().getRectBound(), the same rectangle that ST_Envelope and the other geography joins use. It follows the edges of the geography and reaches a pole when a polygon contains that pole. A polygon whose rings enclose no area has no such rectangle and is skipped like an empty geography.
  • The rectangle distance is reduced by 1e-6 radians (about 6 m) before it is used as a lower bound. S2 computes it through an arc sine that loses up to about 4e-8 radians near 180 degrees, and near 90 degrees from the side of a rectangle, so a smaller margin lets a neighbour be ranked behind one a few centimeters farther away.
  • Geographies are added with add, as items are to an STRtree, and all of them have to be added before the first search. After that the index can be searched from several threads. NULL and empty geographies are skipped on insert and have no neighbours as a query.

How was this patch tested?

New GeographyKnnIndexTest:

  • order and distances of the result
  • comparison with a brute-force ranking for 5000 random points over the whole sphere, for random polygons, and for polygon queries, and for lines, multi-part geographies and geometry collections
  • nearest neighbour across the antimeridian, an object that crosses the antimeridian, nearest neighbour across a pole, and a latitude where planar degrees give a different order
  • minimum distance to a polygon rather than distance to its centroid
  • k larger than the number of objects, k below 1, ties with and without includeTies, NULL and empty objects and queries, an empty index, adding after the first search
  • polygons that contain a pole
  • comparison with a brute-force ranking for objects within centimeters to meters of the antipode of the query, and for objects about 90 degrees away from it
  • merge, Java serialization of the neighbours, bounds and toEnvelope
  • distanceLowerBound against the distance to random geographies inside the envelope, and across the antimeridian

Replacing the rectangle distance with a planar one makes several of these tests fail, and removing the margin on the lower bound makes the two precision tests fail. The tests also pass with a default locale that writes a decimal comma.

Did this PR include necessary documentation updates?

  • No, this PR does not affect any public API so no need to change the documentation.

GeographyKnnIndex ranks geographies by their minimum distance on the
sphere. It stores each geography's latitude / longitude bounding
rectangle in an STRtree and runs a best-first search that prunes with
the distance on the sphere between rectangles, so it stays correct
across the antimeridian and near the poles.

The index lives in the common module with a generic item type, so that
the Spark joins and other engines can share it.
The bounding rectangle of a geography came from Geography.region(),
while distances are measured on the shapes behind Functions.distance.
The two can read a WKB polygon differently: a ring along the equator
was bounded as one hemisphere and measured as the other, and a polygon
whose rings enclose no area had no bounds at all although it has a
distance. The rectangle is now computed from the shapes that distance
uses, edge by edge, and extended to the poles a polygon contains.

The rectangle distance loses precision near 180 degrees, and near 90
degrees from the side of a rectangle, by up to about 4e-8 radians. That
is more than the 1e-9 radians it was reduced by, so a neighbour could
be ranked behind one a few centimeters farther away. The slack is now
1e-6 radians.

The polygon tests also wrote WKT with the default locale, which fails
to parse where the decimal separator is a comma.
Geographies are added to the index directly, like items to an STRtree,
instead of through a builder. The search loop treats tree nodes and
geographies as the same kind of queue entry, and the bounding rectangle
code drops guards it does not need.
The index now takes the bounding rectangle of a geography from
Geography.region(), as ST_Envelope and the other geography joins do,
instead of computing it from the shapes of the distance function.
@jiayuasu jiayuasu added this to the sedona-2.0.0 milestone Nov 17, 2026
@jiayuasu
jiayuasu merged commit 032d05b into apache:master Nov 17, 2026
45 of 46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant