All findings · Lucene finding · LFV-737 · geoexactcircle-outside-distance-ignores-slice-end-points

GeoExactCircle outside distance never measures the slice end points, so points due north or south of an exact circle on WGS84 can get +Infinity

On WGS84, a geo3d exact circle's computeOutsideDistance can return +Infinity for points due north or south of its centre: for a 637 km circle, every such point tested, such as one 61 km beyond the edge. Its slice joints are never measured; every release since 7.1.0 has it.

Summary

Priority
Low, rank 404GeoExactCircle.computeOutsideDistance returns +Infinity or the far-side value at slice joints, but only direct callers of the experimental exact circle can hit it because no Lucene query or sort uses it.
Impact
silent wrong results
Since
7.1.0 (LUCENE-7970, 2017)introduced 2017-09-25 · about 9 years
Affected
7.1.0 to 10.5.2 and main
Status
documented
Found by
a bounded-exhaustive comparison of geo3d distance functions against a reference

AI-assisted page: this finding was first signalled by a semi-formal check (an exact check of the real jar, or a modelling hypothesis that a tool then witnessed) and then checked with AI tools; the checks it went through, and what each found, are under Prior reports, history and checks. Not yet triaged by a human. Snapshot of 2026-11-03.

The finding in 30 seconds

geo3d's exact circle is a ring of flat slices glued at their end points. Its outside distance is the smallest of the slices' distances, and each slice only counts points of its own sector. A joint between two slices is in neither sector's interior and is never measured, so a point whose nearest edge point is a joint can get +Infinity.

Diagram of the repro: a WGS84 exact circle with centre (0.3, 0.2) and radius 0.1, approximated by 8 slices that meet at joints at bearings 0, 45, ..., 315 degrees; a point q 0.11 due north of the centre, outside the circle; its nearest edge point is the north joint, 0.009558 away; the two slices that meet there each find their closest point just past the joint, outside their sector, and return +Infinity; Lucene's outside distance is +Infinity, and with the fix (also measuring the slice end points) it is 0.009558.
The repro's WGS84 exact circle as slices that meet at joints, and the point q 0.11 due north of the centre: its nearest edge point is the north joint, which neither neighbouring slice measures, so Lucene returns +Infinity instead of 0.009558. Select the still to open it at full size. Generated with AI, checked against the code.
  1. You build an exact circle on WGS84: makeExactGeoCircle(WGS84, 0.3, 0.2, 0.1, 1e-6), radius 0.1 (about 637 km), accuracy 1e-6 (about 6 m). Lucene builds it from 8 flat slices that meet at bearings 0, 45, 90, ..., 315 degrees from the centre.
  2. You ask computeOutsideDistance(DistanceStyle.ARC, q) for q = (0.41, 0.2), 0.11 due north of the centre. You expect at most 0.009558 (about 61 km): the shape's own edge on that meridian is that far from q.
  3. You get Infinity, with assertions on or off, for all six points tested due north or south of this circle, in all five plane-based distance styles. 3 km east, at (0.41, 0.2005), you get 0.009559, within 4e-9 of the edge point there.
  4. Why: each slice measures only to its own sector of the edge. q's nearest edge point is the north joint, where two sectors meet; each of those slices finds its closest point just past the joint, outside its sector, and declines. Nobody measures the joint. (A joint alone is not always enough: a foot that misses by less than the 1e-12 tolerance is kept.)
  5. Near other joints the far side of the circle can win: 0.2504 instead of 0.0501 at bearing 135.05 degrees. On the unit sphere, a point 0.001 due west of a circle of radius 1e-4 gets 0.0012, the offset plus twice the radius.

Who is affected

Only applications that call the experimental geom API directly: an exact circle from GeoCircleFactory.makeExactGeoCircle and its computeOutsideDistance. No Lucene query, sort or module does. For them it is deterministic: on WGS84, points on the centre's meridian outside the circle got +Infinity in every check at radius 0.01 to 0.4 (64 to 2,548 km).

Mechanism, terms and numbers1,049 words

GeoCircleFactory.makeExactGeoCircle builds a circle on an ellipsoid out of flat slices that meet at their end points. GeoExactCircle.outsideDistance takes the minimum, over the slices, of the distance to the slice's plane bounded to the slice's own sector, and never measures the end points. When the nearest edge point of an outside point is such a joint, the point's foot on each of the two neighbouring slice planes can land just past the joint, outside that slice's sector; then both return +Infinity and the minimum is +Infinity or the distance to a slice on the far side of the circle. On WGS84 two joints always sit due north and due south of the centre: for the exact circle at (0.3, 0.2) with radius 0.1 (about 637 km) and accuracy 1e-6, a point 0.11 due north gets +Infinity instead of at most 0.009558 (the shape's own edge point), and so do the other five points tested due north or south of this circle, in all five plane-based distance styles, while the same point 0.0005 (about 3 km) east gets 0.009559, within 4e-9 of its edge point. In a sweep of WGS84 and CLARKE_1866 circles of radius 0.01 to 0.4, every check on the centre's meridian gave +Infinity; at radius 0.001 some did not. Near other joints the far side can win (0.2504 instead of 0.0501), and on the unit sphere a circle of radius 1e-4 gives 0.0012 instead of 0.001 for a point due west. Only applications that call the experimental geom API directly are hit; no Lucene query, sort or module computes an outside distance on an exact circle. The defect is as old as the class (LUCENE-7970, 7.1.0); adding the stored end points to the minimum, as GeoConvexPolygon does with its vertices, closes the gap, with all of lucene/spatial3d passing; outside distances on an ellipsoid stay approximate.

Terms

  • Exact circle: GeoCircleFactory.makeExactGeoCircle(planetModel, lat, lon, radius, accuracy) returns a GeoExactCircle, "the most accurate shape to represent a circle in non-spherical planets". It approximates the circle by slices: each slice is a plane through three points of the true circle, bounded to its sector (the wedge between its two end points, seen from the centre) by two sided planes through the centre. Slices are split until every plane is within accuracy of the circle.
  • Joint: an end point shared by two neighbouring slices. The slices start as north -> east -> south and south -> west -> north, so on WGS84 and on the sphere the north and south points are always joints. The repro's circle has 8 slices with joints at bearings 0, 45, 90, ..., 315 degrees.
  • Outside distance: GeoOutsideDistance.computeOutsideDistance(style, point), "the distance from a point to the closest outside edge", 0.0 inside. Radii and ARC distances are in planet units, about radians: 0.1 is about 637 km on Earth.

The bug (main a9779ef3c193)

  1. GeoBaseMembershipShape.computeOutsideDistance returns 0.0 inside, else outsideDistance (GeoBaseMembershipShape.java:46-53).
  2. GeoExactCircle.outsideDistance (GeoExactCircle.java:245-258) returns the minimum over the slices of distanceStyle.computeDistance(planetModel, slice.circlePlane, x, y, z, slice.plane1, slice.plane2): the plane distance, bounded to the slice's sector. No end point is measured.
  3. For ARC that is Plane.arcDistance (Plane.java:600-636): intersect the slice plane with the plane through the point perpendicular to it, keep the intersection points that meet all bounds, else +Infinity.
  4. Think of the slices as the sides of a polygon. For a point beyond a corner, the nearest boundary point is the corner, and the foot on each of the two sides that meet there lies past the corner, outside that side. Each side declines, and the corner is never asked. For q 0.11 due north, all 8 slices find intersection points and none is inside its sector; on the two slices that meet at the north joint the near foot (0.00955806 away) fails the bound through the joint by 6.85e-6, a geometric miss, not rounding. The joint is 0.009558 away. A joint as the nearest edge point is not enough on its own: a foot that misses by less than the 1e-12 tolerance is kept (the independent reproducer found 5 of 6 sphere joint cases wrong), and at radius 0.001 on WGS84 some meridian points get finite values.
  5. If the perpendicular plane meets a slice on the far side inside its sector, that distance wins: about the offset plus twice the radius. On the sphere a small circle lies in one plane, so the slice planes nearly coincide and the near foot misses its bound only by rounding (about 3e-12 to 4e-12): for q 0.001 west of a 1e-4 circle, the two slices at the west joint return +Infinity and the two eastern ones 0.0012.
  6. The class stores each slice's end points (notableEdgePoints, GeoExactCircle.java:437-454) and uses them in getBounds (:306-316) and intersects (:277-304), but not here. GeoConvexPolygon.outsideDistance adds its vertices (GeoConvexPolygon.java:511-528) and GeoRectangle.outsideDistance its four corners (GeoRectangle.java:325-347).

The numbers (executed; assertions on and off give the same values)

exact circlepointLucenecorrect
WGS84 (0.3, 0.2), r 0.1, accuracy 1e-60.11 due north (0.41, 0.2)Infinity (all 5 plane-based styles)0.009558 (the end point; the fix gives it)
same0.15, 0.2 north; 0.11, 0.15, 0.2 southInfinityat most 0.049558, 0.099558; 0.009501, 0.049501, 0.099501
same (control)(0.41, 0.2005), 3 km east of the meridian0.009559at most 0.009559 (its edge point, 4e-9 from the value)
WGS84, r 0.1about 0.05 outside at bearing 135.05 degrees0.2504 (far side)0.0501
SPHERE (0, 0.2), r 1e-4, accuracy 1e-60.001, 0.01, 0.1 due west0.0012, 0.0102, 0.10020.001, 0.01, 0.1
SPHERE, same circle (control)bearings 0.37, 2.1, 4.4the offsetthe offset

On WGS84 and CLARKE_1866, meridian points outside the circle got +Infinity in every check at radius 0.01 to 0.4 (64 to 2,548 km), and in 28 of 48 per model at radius 0.001; around bearings 0 and 180 the +Infinity wedge is about +/-0.004 degrees wide for a point 0.01 outside a 637 km circle, +/-0.015 at 0.05 and +/-0.0225 at 0.1 (slightly narrower around 180). On the sphere only some very small circles at an exact joint bearing are hit: the finder's sweep found them at its radii 1e-5 and 1e-4, none at 1e-6 or at 1e-3 and above, and another sweep none at 3e-5.

Exposure and likelihood437 words
  • Who. Applications that build a circle with GeoCircleFactory.makeExactGeoCircle and call computeOutsideDistance on it themselves, for example to rank or filter points by their distance to a circular area. The Geo3DDocValuesField outside-distance sorts build standard circles (Geo3DUtil.fromDistance -> makeGeoCircle, Geo3DUtil.java:160), and spatial-extras builds exact circles (Geo3dShapeFactory.java:188-194) but never calls computeOutsideDistance.
  • Effect. A point just beyond the circle's north or south edge can look infinitely far away; near some other joints it looks about twice the radius too far. No exception, with assertions on or off; query-time arithmetic only, nothing is written and no hit set changes.
  • Default vs opt-in. The optional lucene/spatial3d module and its @lucene.experimental geom API. Within it no special setting: WGS84 and CLARKE_1866 are hit at every accuracy tested (1e-4 and 1e-6 in the audit sweep; 1e-4 to 1e-8, by factors of 10, for the repro's circle in a fact-check probe), the sphere only for very small circles.
  • Scale. None: one point, deterministic.
  • How likely. Rare for arbitrary data (3 to 21 of 72,001 bearings for a 637 km circle at accuracy 1e-6, 73 at 1e-4), but likely for a hand-written "point due north" check (every one at radius 0.01 to 0.4 in the audit sweep). No public report in the 9 years since 7.1.0.
  • Could widen. Open issue #9623 (LUCENE-8577) proposes exact circles for Geo3DPoint distance queries. The query itself uses membership only (Geo3DPoint.java:122-129), but if the change were made in the shared Geo3DUtil.fromDistance, Geo3DDocValuesField.newOutsideDistanceSort (Geo3DDocValuesField.java:249-258) would reach this bug.
  • Workaround. A standard circle (GeoCircleFactory.makeGeoCircle) has no slices and so no joints: with the same centre and radius it gave a finite 0.05 where the exact circle gave +Infinity in a cross-model check (not tried as a workaround beyond that). It is a less exact shape on an ellipsoid, and below about 50 km it hits LFV-274. A caller cannot add the joints itself: getEdgePoints exposes only one edge point (GeoExactCircle.java:193).
  • Related findings (separate reports, other fix sites): LFV-274 (on an ellipsoid, circles smaller than about 50 km, standard and exact, get +Infinity and somewhat larger ones finite values that are too large, because the perpendicular plane in Plane.arcDistance misses the circle or meets it at the wrong points; narrow fix in GeoStandardCircle; this fix also makes its small exact-circle +Infinity values finite, though not exact), and two other outside-distance defects in sibling shapes: LFV-782 (the outside distance to a polar box ignores the pole, a missing candidate point as here) and LFV-735 (GeoStandardPath measures its end caps to the end point instead of the cap edge).

Minimal repro

The test shows that GeoExactCircle.computeOutsideDistance returns +Infinity for a point due north of a WGS84 exact circle, because it never measures the slice end point (the joint where two slices meet) that is the closest edge point.

TestGeoExactCircleOutsideDistance.java (JUnit test)29 lines
/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.apache.lucene.spatial3d.geom;

import org.apache.lucene.tests.util.LuceneTestCase;

public class TestGeoExactCircleOutsideDistance extends LuceneTestCase {

  public void testPointDueNorthOfCentre() {
    GeoCircle circle = GeoCircleFactory.makeExactGeoCircle(PlanetModel.WGS84, 0.3, 0.2, 0.1, 1e-6);
    GeoPoint point = new GeoPoint(PlanetModel.WGS84, 0.41, 0.2);
    // expected: 0.009558 to the circle's north edge point (latitude 0.400442); actual: +Infinity
    assertEquals(0.009558, circle.computeOutsideDistance(DistanceStyle.ARC, point), 1e-6);
  }
}
Command
# in a copy of this finding's repro folder (repro/LFV-737/, which holds src/)
# Lucene checkout at the tested commit, built with: ./gradlew --no-daemon assemble   (JDK 25)
LUCENE=/path/to/lucene; V=12.0.0-SNAPSHOT
CP=$LUCENE/lucene/core/build/libs/lucene-core-$V.jar
CP=$CP:$LUCENE/lucene/codecs/build/libs/lucene-codecs-$V.jar
CP=$CP:$LUCENE/lucene/test-framework/build/libs/lucene-test-framework-$V.jar
CP=$CP:$LUCENE/lucene/spatial3d/build/libs/lucene-spatial3d-$V.jar
CP=$CP:randomizedtesting-runner-2.9.2.jar:junit-4.13.2.jar:hamcrest-3.0.jar   # test-framework dependencies (Maven Central)
CP=$CP:junit-platform-console-standalone-6.1.3.jar   # JUnit Platform console launcher (Maven Central)

javac -d classes -cp "$CP" $(find src -name '*.java')

# run 1: assertions on, seed DEADBEEF
java -ea -esa --add-modules jdk.incubator.vector --enable-native-access=ALL-UNNAMED -Dtests.seed=DEADBEEF \
     -cp "classes:$CP" org.junit.platform.console.ConsoleLauncher \
     execute --select-class org.apache.lucene.spatial3d.geom.TestGeoExactCircleOutsideDistance
Expected vs actual
Expected: about 0.009558, the ARC distance from (0.41, 0.2) to the circle's north edge point. Actual (-ea and -da): +Infinity.

All files of this repro: repro/LFV-737/

More files, environment, full output34 words

Environment

luceneLucene main e1faeba08bde (2026-11-02)
jdk25.0.3+9-LTS

Observed output

# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
    => java.lang.AssertionError: expected:<0.009558> but was:<Infinity>
       org.junit.Assert.fail(Assert.java:89)
       org.junit.Assert.failNotEquals(Assert.java:835)
       org.junit.Assert.assertEquals(Assert.java:555)
       org.junit.Assert.assertEquals(Assert.java:685)
       org.apache.lucene.spatial3d.geom.TestGeoExactCircleOutsideDistance.testPointDueNorthOfCentre(TestGeoExactCircleOutsideDistance.java:27)

Why it looks unintended

  • GeoOutsideDistance promises "the distance from a point to the closest outside edge"; a slice joint is a point of that edge.
  • The class uses its end points in getBounds and intersects; LUCENE-8057 fixed the same omission in getBounds as a bug.
  • Not an approximation error: 3 km east of the joint the value is within 4e-9 of the edge.
The full argument224 words
  • The contract. GeoOutsideDistance: "Implemented by Geo3D shapes that can compute the distance from a point to the closest outside edge." and "A return value of 0.0 should be returned for points inside of the shape." (GeoOutsideDistance.java:19-21, :31-33). +Infinity is documented only for the internal distance of points outside (GeoDistance.java:28-36).
  • The factory's promise. "the most accurate shape to represent a circle in non-spherical planets", with the accuracy "the maximum linear distance between any point on the surface circle and planes that describe the circle" (GeoCircleFactory.java:49-57). That accuracy bounds how far the planes are from the circle, not the output of computeOutsideDistance, but +Infinity or twice the radius is something the shape's geometry does not explain.
  • The class's own geometry. notableEdgePoints holds every slice's end points and getBounds and intersects use them; only outsideDistance leaves them out.
  • Precedent in the same class. LUCENE-8057 (#9105): 9776e1f4a0a "getBounds() for exact circle did not include segment endpoints.", listed under bug fixes (lucene/CHANGES.txt:7753).
  • Siblings. GeoConvexPolygon and GeoRectangle, built from the same bounded plane distances, add their vertices or corners to the minimum.
  • Tests. No upstream test asserts an outside distance on an exact circle, so nothing pins the current values; the circle assertions in lucene/spatial3d/src/test/org/apache/lucene/spatial3d/geom/TestGeoCircle.java:66-72 use a standard circle and expect the finite distance to the nearest edge.
The devil's-advocate check194 words

The strongest case for "intended" (built by the intended-behaviour checker; rated weak by the cross-model reviews, the independent reproducer and the adjudication): the geom API is @lucene.experimental; GeoExactCircle is package-private and documented as an approximation by sectors; +Infinity is Plane's established "not in bounds" result; GeoOutsideDistance never literally promises a finite value; and in LUCENE-7970 (#9019, 2017-09-23) the geo3d maintainer wrote "GeoOutsideDistance we can't really do at the moment".

Why it fails: the interface promises the distance to the closest edge, and a joint is a point of that edge; the class implements outsideDistance and, 3 km from the joint, is within 4e-9 of the edge distance, so this is a missing case, not a design; +Infinity and +2r are not approximation errors at accuracy 1e-6; the 2017 remark was about an earlier inner/outer-plane design and came two days before the committed design, which implements outside distance; and the maintainers fixed the same end-point gap in getBounds as a bug. Expect a maintainer to say the API is experimental and rarely used, and that the fix changes existing values and adds two point distances per slice: that bears on priority, not validity.

Suggested fix

Recommended Also measure each slice's end points

Also take the minimum over each slice's two end points (slice.notableEdgePoints, already stored and used by getBounds), as GeoConvexPolygon does with its vertices: 8 added lines in GeoExactCircle.outsideDistance. The repro passes with assertions on and off, and all of lucene/spatial3d passes (184 tests including the repro). It closes the joint gap; outside distances on an ellipsoid stay approximate.

All options and trade-offs4 options ยท 925 words

Option A was applied in a scratch worktree at a9779ef3c193 and run through gradle (seed DEADBEEF); B, C and D were not run.

Also measure each slice's end points (validated by the checks) (recommended)

In GeoExactCircle.outsideDistance (:245-258), inside the loop over the slices, also take the minimum of distanceStyle.computeDistance(endPoint, x, y, z) over slice.notableEdgePoints. The boundary is the union of the slice arcs, each closed by its two end points; for an outside point the nearest boundary point is inside some arc or an end point. The bounded plane distance covers the first case (on the sphere; on an ellipsoid only approximately, see the trade-offs and LFV-274), and this adds the second. It uses the same computeDistance(GeoPoint, x, y, z) as the polygon vertices, so every distance style works; the end points lie on the circle, so the change can only lower a value, to the distance of a real edge point.

--- a/lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoExactCircle.java
+++ b/lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoExactCircle.java
@@ -253,6 +253,14 @@ class GeoExactCircle extends GeoBaseCircle {
       if (distance < outsideDistance) {
         outsideDistance = distance;
       }
+      // The closest edge point can be a slice end point, where two slices meet; the bounded plane
+      // distance of neither slice reaches it.
+      for (final GeoPoint endPoint : slice.notableEdgePoints) {
+        final double endPointDistance = distanceStyle.computeDistance(endPoint, x, y, z);
+        if (endPointDistance < outsideDistance) {
+          outsideDistance = endPointDistance;
+        }
+      }
     }
     return outsideDistance;
   }

Trade-offs

  • Cost and compatibility: two extra point distances per slice and call (each joint is measured twice, once per neighbouring slice); no benchmark was run. No API, format or index change, but observable values change: results that are +Infinity or the far side today become finite and smaller. A value changes only where an end point is closer than every foot the slices keep: the joint cases and, on small ellipsoid circles, LFV-274's no-intersection cases at any bearing (with the end points the finder's WGS84 sweep has no +Infinity left).
  • No precision claim: it does not make outside distances on an ellipsoid exact. Away from the joints the bounded plane distance can still exceed the shape's own edge distance on WGS84 (circles at (0.3, 0.2), points 1e-4 to 0.1 outside): by up to 9.5e-6 for the repro's circle (r 0.1), up to 1.6e-4 for a 64 km circle (r 0.01), and at r 0.001 up to 5.3e-4, or 7.1e-4 where the perpendicular plane meets no slice. That is Plane.arcDistance's foot on an ellipsoid (LFV-274's family), not the plane approximation and not this bug; the finder's WGS84 sweep with the end points still has 30,085 finite violations of its bound in 322,675 checks, 9,000 of them where the perpendicular plane meets no slice.
  • With the PR: add the repro (or a joint case in TestGeoExactCircle, which has no outside-distance test yet) and a CHANGES entry.

What the checks validated

  • Stock main a9779ef3c193 (gradle, seed DEADBEEF): the repro fails 2 of 3 with assertions on and off; TestGeoExactCircle (14 tests) and TestGeoCircle (7) pass.
  • With the fix: the repro passes 3 of 3 with assertions on and off; TestGeoExactCircle and TestGeoCircle pass; all of :lucene:spatial3d:test with the repro class added passes (20 classes, 184 tests, 0 failures, 3 skipped); ecjLintMain, ecjLintTest and checkGoogleJavaFormat pass for spatial3d; fix.diff applies cleanly to a clean tree.
  • With the fix shadowed onto the main jars (classpath runner, -da): the repro 3 of 3 (also -ea), the independent reproducer's JUnit 3 of 3 and the cross-model stage's five-style JUnit 4 of 4 pass (2 of 3 and 3 of 4 fail without it).
  • In the hunt, with the same logic: 41 upstream tests (TestGeoExactCircle, TestGeoCircle, TestRandomGeoShapeRelationship, TestGeo3DPoint) pass. The joint sweep (1,701 exact circles on stock) was re-run with the end points on its 567 sphere and 567 WGS84 circles (CLARKE_1866 was not re-run): the sphere goes from 945 violations to 0 in 188,037 checks, and WGS84 from 56,469 +Infinity values in 322,675 checks to none, with 30,085 finite violations left (see the trade-offs). The +Infinity count includes small-radius cases of LFV-274's mechanism (not re-run independently).

Measure each joint once

Measure only endPoint1 of each slice. Every joint is the endPoint1 of exactly one slice: the first two slices start at opposite cardinal points (north and south on WGS84 and the sphere, GeoExactCircle.java:119-135; east and west when zScaling > xyScaling, :103-118), each one's endPoint2 is the other's endPoint1, and each split keeps the parent's endPoint1 for its first half and gives the middle point to the second half as endPoint1 (:169-189).

Trade-offs

Half the extra work, but it relies on that construction invariant and treats notableEdgePoints differently from getBounds and intersects (from reading the code; not run).

Document the limitation

Say in makeExactGeoCircle or GeoExactCircle that outside distances near slice joints may be +Infinity.

Trade-offs

Cheaper to review and a defensible choice for an experimental API with no in-tree caller, but it leaves an unbounded error for legal input when an eight-line fix exists, keeps the result at odds with the closest-edge contract, and callers cannot tell where the joints are.

Not a fix: a tolerance on the sector bounds

Accept feet slightly outside the sector.

Trade-offs

Hides the sphere's rounding case (misses of about 1e-12), but on WGS84 the miss is geometric and grows with the point's distance outside (6.85e-6 at 0.0096, 3.82e-5 at 0.0995 in the audit probe): those two points would need tolerances far above rounding that grow with the distance (no tolerance variant was run).

Downstream note

Downstream relevance: an application that ranks or filters by an exact circle's outside distance can see points just beyond its north or south edge as infinitely far; no Lucene query, sort or hit set is affected.

How it was found

A bounded-exhaustive comparison of geo3d's distance functions against a reference flagged exact-circle outside distances of +Infinity on WGS84 and the far side on the sphere. A sweep over 1,701 exact circles at their slice-joint bearings traced them to the joints: the slice end points removed every sphere violation and every WGS84 +Infinity (CLARKE_1866 not re-run).

The method in full275 words

Method: bounded-exhaustive translation validation on the stock jar (semi-formal tier), in a hunt unit on geo3d distance functions against a reference.

  • The unit's harness compared the circle family's distances with a reference and flagged exact-circle outside distances equal to the reference plus 2r on SPHERE and +Infinity on WGS84.
  • A joint enumerator then built 1,701 exact circles (3 planet models x 7 latitudes x 3 longitudes x 9 radii x 3 accuracies) and checked the slice-joint bearings (all of them for circles with fewer than 32 joints; otherwise every (n/16)-th joint, 16 to 24 of them, plus the joints nearest the four cardinal bearings) +/- {0, 1e-9, 1e-7, 1e-5, 1e-3} and 36 generic bearings at 5 outside offsets each, against an upper bound from the shape's own isWithin. A per-slice classifier showed that the perpendicular plane does meet each slice plane but the foot misses the slice's sector bounds.
  • With the slice end points added in the harness: SPHERE from 945 violations to 0 in 188,037 checks, WGS84 from 56,469 +Infinity values in 322,675 checks to none. The WGS84 count includes small-radius cases of LFV-274's no-intersection mechanism, which the end points also make finite. 30,085 finite WGS84 violations remain, 9,000 of them where the perpendicular plane meets no slice: they come from the plane distance on an ellipsoid (LFV-274's family), not from the joints. CLARKE_1866 was not re-run with the end points.
  • Found 2026-10-27 on main a9779ef3c193 and reduced to a public-API JUnit; then checked by an independent reproducer, an intended-behaviour devil's advocate, reviews by a model from another family (OpenAI GPT-5.6), an adjudication, a history, prior-report and security follow-up, and a credibility audit.

Prior reports, history and checks

Prior reports

No prior report found (2026-10-27): 179 recorded queries over GitHub issues, PRs and commits, the JIRA archive, the mailing lists, Elasticsearch, OpenSearch, Solr and the web, plus all 410 open PRs. Closest: LUCENE-8057 (#9105), which fixed the same end-point omission in this class's getBounds() in 2017.

Searches3 searches

Verdict new: no report by anyone. 27 hits were read, none the same bug. Limits: 167 of the 179 records completed; five GitHub queries hit rate limits and were re-run, one web query was throttled, six web queries of the tool are recorded as pending (not run), list search is keyword based, and private reports are not visible.

What was searched, when, and what turned up
WhereWhenHowFound
apache/lucene issues, PRs and commits (incl. migrated JIRA); Elasticsearch, OpenSearch, Solr; dev@, java-user@, issues@, commits@, solr-user@, users@solr; the JIRA archive; the web2026-10-27the hunt's prior-report tool in three variants (121 queries), 34 manual GitHub queries, 24 other manual records27 hits read, none the same bug
Closest items (read)2026-10-27issue bodies and comments#9105 (LUCENE-8057: the same end-point omission, fixed in getBounds() only); #9019 (LUCENE-7970: added the class, pre-design remark); #9102, #9104, #9114, #9128 (construction and bounds, never outside distance); open #9623 (LUCENE-8577: would widen the reach)
all 410 open apache/lucene PRs2026-10-27changed-file scan for spatial3d pathsnone touches GeoExactCircle, GeoBaseCircle, GeoCircle, Plane or GeoOutsideDistance; #14775 (inverse sort) leaves distance values alone

History

There since the class was added: LUCENE-7970 (2c95fa4118c / 2c79710618e, 2017-09-25, first released in 7.1.0) never measured the slice end points. Executed on 12 releases from 7.1.0 to 10.5.2 and on main, all affected; not fixed upstream.

Versions tested7 rows

Introduced by LUCENE-7970 (#9019, opened by @iverase): "LUCENE-7970: Add an exact version of a circle, which uses Vicenty logic to construct out of planes.", committed by @DaddyWri on 2017-09-25 as 2c95fa4118c (branch_7x) and 2c79710618e (master); no pull request. First release 7.1.0.

On 7.1.0 the repro's exact inputs pass only because that circle was built from wrongly interpolated slices; at accuracy 1e-4 the same WGS84 test gives +Infinity for 6 of 6 meridian points there. The repro's WGS84 inputs first fail with LUCENE-8054 (a7e6df03fc0, 7.2.0) and its sphere inputs with LUCENE-8080 (f79baa044e4, 7.3.0), which moved those points onto joints; the 7.2.0 sector-bounds rework (LUCENE-8056/8066) changes no result (commit-level bisection). Upstream main a95c754ee003 has the same code.

Versions and branches executed, with results
Version or commitVerdictResult
main a9779ef3c193 affectedthe repro fails 2 of 3 through gradle (assertions on and off) and the classpath runner (-ea, -da); passes with the fix
7.3.0, 7.7.3, 8.0.0, 8.11.4, 9.0.0, 9.1.0, 9.12.3, 10.0.0, 10.5.2 affectedthe hunt's JUnit fails 2 of 2 with the same values (Infinity on the meridian; 0.0012 on the sphere)
7.2.0, 7.2.1 WGS84 inputsthe WGS84 meridian points give Infinity; the sphere inputs are not yet on a joint
7.1.0 accuracy 1e-4the repro's inputs pass, but accuracy 1e-4 gives Infinity for 6 of 6 meridian points (-ea and -da)
7.1.0 with the end points added controlfinite values, the ones main gives with the fix
7.0.1 no APIno makeExactGeoCircle (compile error)
2c95fa4118c (LUCENE-7970) vs parent 1650a11bdeb introducingalready fails 2 of 6 at accuracy 1e-4, 6 of 6 from the next commit; the parent has no exact circle

Independent checks

An independent reproducer and an intended-behaviour devil's advocate reproduced it with their own tests and controls; reviews by a model from another family (OpenAI GPT-5.6) reproduced it and rated the intended-behaviour case weak; an adjudication and an audit re-ran it with new seeds, and this doc's gradle runs reproduce it on a9779ef3c193.

Checks and verdicts509 words

Every executing check wrote or re-ran its own repro on stock main; every control passed and every fix check passed. Corrected along the way: the earlier "7.2.0 regression" reading (from code reading) gave way to the executed history, which shows the defect since 7.1.0. Not independently re-run: the finder's sweep counts.

Each independent check and its verdict
CheckVerdictWhat it found
finder candidateJoint sweep over 1,701 exact circles (with the end points added: 0 sphere violations and no +Infinity on WGS84; CLARKE_1866 not re-run), then a 2-test public-API JUnit: fails on main (-ea, -da), 10.5.2 and 9.12.3; passes with the fix.
cross-model review, refute lens (OpenAI GPT-5.6) real, highIts own test: +Infinity at the WGS84 north joint where a boundary point from getEdgePoints is 0.1307 away; off-joint control passes; the fix passes its tests, TestGeoExactCircle and TestGeoCircle; intended-behaviour argument weak.
independent reproducer realOwn JUnit: 12 of 12 meridian checks Infinity (ARC, LINEAR), 5 of 6 sphere joints offset plus 2r, on main (-ea, -da), 10.5.2 and 9.12.3; mechanism probe (1.31e-5 miss); own fix passes.
cross-model review, intended lens (the cross-model review, OpenAI GPT-5.6) real, highOwn main(): WGS84 Infinity where a standard circle gives 0.05; SPHERE 0.0012 vs 0.0010; passes with the fix. A separate test shows Infinity in all five plane-based styles.
intended-behaviour devil's advocate real, highBuilt the strongest case for intended behaviour and rejected it; per-slice probe (8 slices miss their sector, 0 miss the intersection); 72,001-bearing sweeps (wedges, far side at 135.05 degrees); 0 errors with the fix.
adjudication confirmed, highRe-ran the independent reproduction's JUnit with new seeds (2 of 3 fail with -ea and -da, control passes) and with the fix (3 of 3 pass).
the version-history runs confirmed, history correctedCommit-level bisection: the defect dates from LUCENE-7970 (7.1.0), not the 7.2.0 sector rework named by earlier checks.
Registration check and credibility audit keepRe-ran the finder's JUnit (2 of 2 fail, -da, new seed; 2 of 2 pass with the fix); own probe (6.85e-6 miss, intersections exist) and meridian sweep (172 of 192 Infinity per ellipsoid model).
this doc reproduced; fix validatedGradle in a clean worktree at a9779ef3c193: the repro fails 2 of 3 (control passes) with assertions on and off; with the fix 3 of 3 pass and all of lucene/spatial3d passes (184 tests).
independent fact check of this doc real; side statements correctedRe-ran the gradle repro (stock 2 of 3 fail with assertions on, off and a new seed; with the fix 3 of 3 and all of lucene/spatial3d), release runs from 7.1.0 to 10.5.2 and its own probes; found the remaining WGS84 error with the fix (up to 1.6e-4 for a 64 km circle) and four wording errors, all fixed here.
cross-model fact check of this doc (OpenAI GPT-5.6) real; scope and wording correctedRead-only review of the brief against the saved evidence: core bug real, intended-behaviour case weak; its scope, wording and caveat findings were fixed or answered.

Diagram file

The diagram as a still image, generated with AI and checked against the code. It is under 10 MB, the limit for a GitHub attachment.

Sources and provenance

Finding record, Lucene and upstream sources, repro files and diagram notes1,635 words

Finding record

LFV-737, snapshot of 2026-11-03

Registry title
GeoExactCircle.computeOutsideDistance never measures the slice end points where two of its approximation slices meet, so a point outside whose nearest edge point is such a joint gets +Infinity or the distance to the far side of the circle: on WGS84 every point due north or due south of the centre, at any radius; on the sphere a point 0.001 rad due west of a 1e-4 rad exact circle gets 0.0012
Area
spatial3d / geom exact-circle outside distance (GeoExactCircle.outsideDistance, reached through GeoCircleFactory.makeExactGeoCircle and GeoOutsideDistance.computeOutsideDistance)
Priority
Low, rank 404: GeoExactCircle.computeOutsideDistance returns +Infinity or the far-side value at slice joints, but only direct callers of the experimental exact circle can hit it because no Lucene query or sort uses it.
Impact class
silent wrong results
Impact note
Only applications that build a circle with the experimental GeoCircleFactory.makeExactGeoCircle and call computeOutsideDistance on it themselves are hit (no Lucene query, sort or module reaches it: the Geo3DDocValuesField outside-distance sorts build standard circles, and spatial-extras builds exact circles but never asks them for an outside distance), with no scale and no special setting: on WGS84 or CLARKE_1866 every point on the centre's meridian outside the circle gets +Infinity at any radius and any distance, in every plane-based distance style (ARC, LINEAR, LINEAR_SQUARED, NORMAL, NORMAL_SQUARED), hairline wedges around those joints do too (about +/-0.004 to +/-0.0225 degrees of bearing for a 637 km circle), points near other slice joints can get the far-side value (0.2504 instead of 0.0501 at one), and on PlanetModel.SPHERE only very small circles (radius about 1e-5 to 1e-4 rad) at an exact joint bearing are hit, through rounding; the error is unbounded (+Infinity or about twice the radius), query-time only, nothing is persisted, and every release since 7.1.0 has it.
Summary
You build an exact circle on the default WGS84 planet model (GeoCircleFactory.makeExactGeoCircle, centre latitude 0.3 and longitude 0.2, radius 0.1 rad, about 637 km, accuracy 1e-6) and ask computeOutsideDistance for a point 0.11 rad due north of the centre, expecting 'the distance from a point to the closest outside edge' (GeoOutsideDistance), about 0.0096. Instead you get +Infinity, and the same for every point due north or due south at any distance; on the unit sphere a point 0.001 rad due west of a 1e-4 rad exact circle gets 0.0012, the distance to the far side of the circle. The cause: GeoExactCircle.outsideDistance takes the minimum of each slice's plane distance bounded to that slice's sector and never measures the slice end points, so when the nearest edge point is a joint between two slices, the point's foot on each neighbouring plane falls just outside its sector, both return +Infinity, and the minimum is +Infinity or a foot on the far side.
Introduced
2c95fa4118c2a19fc385b23a90facc4bd0bf9006 on branch_7x (the same change as 2c79710618e7b6d8b7ecff5dc17cfe7abbd59434 on master, committed the same minute, identical geom package), 'LUCENE-7970: Add an exact version of a circle, which uses Vicenty logic to construct out of planes.' (LUCENE-7970 = apache/lucene#9019, opened by iverase, committed by DaddyWri; no pull request). It added GeoExactCircle with outsideDistance = the minimum of the bounded plane distances and no plane end points [read: git blame of main GeoExactCircle.java:248 and :253-258 resolves to 2c79710618e; exec: a probe at accuracy 1e-4 already fails 2 of 6 at this commit and 6 of 6 from the next one, 9add14a513a; the parent 1650a11bdeb has no exact circle]. The candidate's own inputs fail only after two construction fixes moved its points onto slice joints [exec, commit-level bisection]: LUCENE-8054 a7e6df03fc0 (2017-11-21, first released in 7.2.0) for the WGS84 test and LUCENE-8080 f79baa044e4 (2017-12-08, 7.3.0) for the sphere test. Not the cause: the 7.2.0 sector-bounds rework LUCENE-8056 (183d6d67bce) and LUCENE-8066 (d4b7eccda30), which earlier readings of the code had named; they change no probe result [exec]. (2017-09-25 (LUCENE-7970); the trigger commits for the candidate's exact inputs are dated 2017-11-21 (LUCENE-8054) and 2017-12-08 (LUCENE-8080))
First affected version
7.1.0 [exec], the first release with GeoExactCircle: the same WGS84 meridian test at accuracy 1e-4 gives +Infinity for 6 of 6 points with -ea and -da, and adding the stored end points to the 7.1.0 code removes it (the candidate's own inputs pass on 7.1.0, because its exact circle there is built from wrongly interpolated slices; they first fail on 7.2.0 for WGS84 and 7.3.0 for the sphere). 7.0.1 and earlier have no makeExactGeoCircle [exec: compile error]
Affected versions
every release from 7.1.0 through 10.5.2 (the last release before 10.6.0) and main a9779ef3c193. Executed (candidate JUnit, a history JUnit with the accuracy-1e-4 case, and a public-API main() probe): 7.1.0, 7.2.0, 7.2.1, 7.3.0, 7.7.3, 8.0.0, 8.11.4, 9.0.0, 9.1.0, 9.12.3, 10.0.0, 10.5.2 and main (-ea, and -da on 7.1.0 and main); from 7.3.0 to main the probe output is identical line for line. [read] the others: outsideDistance is unchanged since LUCENE-8071 (6c3869f8b19, 2017-12-01) apart from the LUCENE-9570 reformatting, and the release tags 7.1.0-7.7.3, 8.0.0-8.8.1, 9.0.0-9.12.3 and 10.0.0-10.5.2 contain the introducing commit (8.8.2-8.11.4 are not tags in the history clone; 8.11.4 executed). Upstream origin/main a95c754ee003 has the same code [read]. Not affected: 7.0.1 and earlier (no API). Never fixed. All runs on JDK 25.0.3.
Status
documented (brief, checks and a per-finding doc exist)
Verification
Confirmed by AI-run checks: an independent repro, a cross-model review that agreed, an intended-behaviour check, a prior-report search and a skeptical audit; not yet triaged by a person.
Found by
semi-formal: exhaustive check
Upstream and prior reports
none found (2026-10-27, read-only; the prior-report record, verdict new: 179 recorded queries (121 from the scripted prior-report search in three variants, 34 manual GitHub queries, 24 other manual records) over apache/lucene issues, PRs and commits (including migrated JIRA issues), Elasticsearch, OpenSearch and Solr, the dev@, java-user@, issues@ and commits@ lists, solr-user@ and users@solr, the JIRA archive, the changed files of all 410 open apache/lucene PRs and the web; 27 hits read, none the same bug). Related, not reports: #9105 (LUCENE-8057, closed 2017), which fixed the same omission of slice end points in GeoExactCircle.getBounds() and left outsideDistance alone; #9019 (LUCENE-7970), which added GeoExactCircle and where a maintainer wrote 'GeoOutsideDistance we can't really do at the moment' before the design that was committed; #9102 (LUCENE-8054), #9104 (LUCENE-8056), #9114 (LUCENE-8066) and #9128 (LUCENE-8080), construction and bounds changes that never discuss outside distance; open #9623 (LUCENE-8577), which proposes exact circles for Geo3DPoint distance queries and would widen the reach if adopted; open PR #14775 (inverse sort), which leaves distance values alone. Not fixed upstream: upstream main a95c754ee003 has the same outsideDistance (last change to the file dfd536186fe, 2026-04-01, #15889, cosmetic), and no open PR touches GeoExactCircle, GeoBaseCircle, GeoCircle, Plane or GeoOutsideDistance. Gaps: five manual GitHub queries failed and were re-run; one of six web queries was throttled and not retried; list search is keyword-based; the file list of open PR #16911 is capped at 3000 files. Update 2026-11-02: #16911 (open when searched) merged 2026-10-29 as 48cf3f515d6, which emptied branch_10x to a README tombstone (10.6 is the last 10.x minor; 10.x fixes go to branch_10_6); it fixes nothing here.
Lucene commit tested
main a9779ef3c193 (commit of 2026-10-26): the repro JUnit through gradle in a clean worktree, stock and with the fix, assertions on and off, plus TestGeoExactCircle, TestGeoCircle and all of lucene/spatial3d (20 classes, 184 tests) with the fix (2026-10-28); through the hunt's classpath runner, the same JUnit on main (-ea, -da, fix shadowed) and on 10.5.2 (-da), and two independent JUnits with and without the fix. History runs on 7.1.0, 7.2.0, 7.2.1, 7.3.0, 7.7.3, 8.0.0, 8.11.4, 9.0.0, 9.1.0, 9.12.3, 10.0.0, 10.5.2 and main, plus a commit-level bisection from 7.1.0 to 7.3.0; the cross-model reviews on main ea57360912ed (identical spatial3d code). OpenJDK 25.0.3, Linux aarch64.

Lucene and upstream sources

Finding record
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoExactCircle.java:245-258 (at a9779ef3c193) (outsideDistance: minimum over circleSlices of distanceStyle.computeDistance(planetModel, slice.circlePlane, x, y, z, slice.plane1, slice.plane2), no end point), :437-454 (CircleSlice: plane1/plane2 = SidedPlane(check, endPoint, center) bound each slice to its sector; notableEdgePoints = {endPoint1, endPoint2}), :94-135 (on WGS84 and SPHERE the first two slices are built north-east-south and south-west-north, so the north and south points are joints), :26-32 (class javadoc: edge approximated by sectors, @lucene.experimental)
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoOutsideDistance.java:19-23 (at a9779ef3c193) ('compute the distance from a point to the closest outside edge', @lucene.experimental) and :31-33 (0.0 for points inside)
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoDistance.java:28-36 (at a9779ef3c193) and :45-49 (+Infinity is documented for the internal distance (computeDistance) of points outside the shape; :59-65 and :76-82 the same for computeDeltaDistance); GeoOutsideDistance has no such clause
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoBaseMembershipShape.java:42-56 (at a9779ef3c193) (computeOutsideDistance: 0.0 if isWithin, else outsideDistance)
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoCircleFactory.java:49-77 (at a9779ef3c193) (makeExactGeoCircle: 'the most accurate shape to represent a circle in non-spherical planets'; accuracy = maximum linear distance between the circle and its planes)
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/Plane.java:600-636 (at a9779ef3c193) (arcDistance: +Infinity when no intersection point meets all bounds)
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoConvexPolygon.java:511-528 (at a9779ef3c193) (sibling shape: adds every vertex distance to the minimum before the bounded edge distances); GeoRectangle.java:325-347 (adds its four corners)
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/Geo3DUtil.java:160 (at a9779ef3c193) (the Geo3DPoint/Geo3DDocValuesField paths build standard circles); lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dShapeFactory.java:189 (builds exact circles; no computeOutsideDistance call anywhere in spatial-extras/src/java) [read: grep, 2026-10-27]
The finding in 30 seconds
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoExactCircle.java:26-33, 94-135, 141-191, 245-258, 277-304, 306-316, 437-454
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoBaseMembershipShape.java:46-53; geom/Plane.java:600-636; geom/GeoOutsideDistance.java:19-24, 31-33; geom/GeoCircleFactory.java:49-80
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoConvexPolygon.java:511-528; geom/GeoRectangle.java:325-347
Who is affected
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/Geo3DUtil.java:160; lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dShapeFactory.java:188-194 (grep: no computeOutsideDistance in spatial-extras/src/java)
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoCircleFactory.java:22; geom/GeoOutsideDistance.java:23 (@lucene.experimental)
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/Geo3DPoint.java:122-129; Geo3DUtil.java:152-160; Geo3DDocValuesField.java:249-258; geom/GeoExactCircle.java:193 (one edge point)
Why it looks unintended
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoOutsideDistance.java:19-21, 31-33; geom/GeoDistance.java:28-36; geom/GeoCircleFactory.java:49-57; geom/GeoExactCircle.java:26-33, 277-316, 437-454
  • lucene/spatial3d/src/test/org/apache/lucene/spatial3d/geom/TestGeoCircle.java:66-72; TestGeoExactCircle.java (no computeOutsideDistance)
Suggested fix
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoExactCircle.java:94-135, 141-191 (slice construction and splitting: endPoint1 of each new slice); geom/GeoConvexPolygon.java:511-528
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoExactCircle.java:103-135 (both branches of the first two slices), :169-189 (the split)

Other statements trace to the hunt's working files (models, logs, reviews and briefs), which are not published.

Repro files

About the diagrams

The caption and alt text describe the bug as the probes record it.

Built 2026-11-03 from the hunt's record of this finding and its doc file. Lucene main under test in this hunt: ea57360912ed. Every listed finding was re-run on Lucene main e1faeba08bde on 2026-11-02.