GeoStandardPath converts a segment point's squared distance back twice: LINEAR_SQUARED and NORMAL_SQUARED return the fourth power (since 9.5.0)

GeoPathFactory.makeGeoPath returns a GeoStandardPath for any width of at least about 3.14e-12 rad (Vector.MINIMUM_ANGULAR_RESOLUTION). Its computeDistance(style, point) adds the pieces of a path distance in "aggregation form" and should convert the sum back once; for LINEAR_SQUARED and NORMAL_SQUARED the aggregation form is the square root and converting back squares it, so the result should be the LINEAR (or NORMAL) distance squared. For a point inside a straight segment the result is converted back twice, once in PathSegment.distance and again in GeoStandardPath.distance, so it comes out as the fourth power: 0.00803 instead of 0.0896 in the repro. Points that lie only in a round cap (at the ends and joints) are converted once and are right; points exactly on a cutoff plane, which lie in both a segment and a cap, can get the fourth power too, so one path jumps 6.3x at a segment's end and can rank a farther point ahead of a nearer one.

The extra conversion is a leftover of a November 2022 refactor (c9c27c755a4, "Make sure use of aggregation form is consistent throughout"), first released in 9.5.0; 9.4.2 is correct. Affected: 9.5.0 to 10.6.0 and main. Reach is narrow: only code that calls the @lucene.experimental geom distance API itself with a squared style; Lucene's path query and path sorts use ARC and are unchanged. Fix: drop the inner conversion (one line, validated).

Since the 2022 refactor, a GeoStandardPath evaluates a binary tree of components: PathNodes whose leaves are PathSegments (the straight corridors) and BaseSegmentEndpoints (the round caps at the ends and joints).

1. Leaves add their pieces in aggregation form. PathSegment.pathDistance returns aggregateDistances(toAggregationForm(across), toAggregationForm(along)) (GeoStandardPath.java:1777-1780); BaseSegmentEndpoint.pathDistance returns toAggregationForm(distance to the cap centre) (:924).
2. The cap leaf stays in aggregation form: BaseSegmentEndpoint.distance returns distanceStyle.aggregateDistances(startingDistance, pathDistance) (:878-886, return at :885).
3. The segment leaf already converts back: PathSegment.distance returns distanceStyle.fromAggregationForm(distanceStyle.aggregateDistances(startingDistance, pathDistance)) (:1538-1549, return at :1547-1548).
4. The tree takes the smaller child: PathNode.distance checks isWithin, then returns Math.min(child1Distance, child2Distance) (:678-687, :686). With step 3 that compares a final-form value with aggregation-form ones; it matters for points exactly on a cutoff plane, which lie in both a segment and a cap.
5. The root converts back again: GeoStandardPath.distance returns distanceStyle.fromAggregationForm(rootComponent.distance(distanceStyle, x, y, z)) (:301-307, :306).
6. For the squared styles that is a second squaring. LinearSquaredDistance.toAggregationForm is Math.sqrt and fromAggregationForm is aggregateDistance * aggregateDistance (LinearSquaredDistance.java:64-70; NormalSquaredDistance.java:64-70 is the same). A point only in a cap is converted back once (d^2), a segment point twice ((d^2)^2). The defaults for ARC, LINEAR and NORMAL are the identity (DistanceStyle.java:116-118, :146-148), so nothing else notices.

SPHERE path (0, 0) -> (0, 0.5), width 0.1, point (0.05, 0.25) in the segment. LINEAR = 0.29934425859988006. In LINEAR_SQUARED the segment leaf adds the square roots of its pieces (sum 0.2993) and squares: 0.08960698515671187, the right answer. The root squares that again: 0.00802941178887518. A start-cap point (0, -0.05) goes through the cap leaf and the root only: LINEAR 0.0499948, LINEAR_SQUARED 0.00249948, right.

The fix that removes only the segment's conversion makes the repro pass, and the whole spatial3d test suite still passes.

• Silent wrong value. A finite, plausible number, about 11x too small at the repro point (0.00803 for 0.0896), no exception. One path's squared distance jumps 6.3x across a segment's end (0.0249 to 0.158 for centre-line points 0.0002 radians apart) and can rank a farther point first: a segment point with LINEAR 0.09996 gets 0.0000998, a cap point with LINEAR 0.0500 gets 0.0025 (an observed pair: a swap needs the segment point's LINEAR^4 to fall below the cap point's LINEAR^2).
• How often. On 400 random 2-4 point paths (SPHERE and WGS84, widths 0.01-0.16), 3,827 of 14,587 in-path points were consistent for both squared styles; all 21,520 mismatches were fourth powers (within the verifier's 1e-9 relative tolerance). The share depends on the sampling: points were drawn within 1.5 widths of a random path vertex. Points exactly on a cutoff plane, such as the vertex where two segments join, get the fourth power too when LINEAR is below 1 (probed: lon 0 and 0.5 on the repro path, the joint of a path lon 0 -> 0.5 -> 1.0), because Math.min compares the segment's converted value with the cap's unconverted one; with the fix they get LINEAR^2.
• Who is exposed. Applications that call GeoPath or GeoDistanceShape.computeDistance with DistanceStyle.LINEAR_SQUARED or NORMAL_SQUARED, on a path from GeoPathFactory.makeGeoPath with a width of at least about 3.14e-12 rad (a GeoStandardPath). The geom API is @lucene.experimental (GeoPath.java:22, GeoDistance.java:24, DistanceStyle.java:22, GeoPathFactory.java:25).
• Lucene itself is not. The squared constants have no caller in Lucene beyond their declarations (DistanceStyle.java:38, :46). The Geo3D distance comparators pass DistanceStyle.ARC (Geo3DPointDistanceComparator.java:85, :131, :173-174; Geo3DPointOutsideDistanceComparator.java:101, :143-144), and Geo3DPoint.newPathQuery matches by shape. Executed: the path query and both path sorts give an identical hit, order and value fingerprint with and without the fix, and points outside a path still get +Infinity in every style.
• Not affected: ARC, LINEAR, NORMAL (identity conversions); computeDistance of the zero-width GeoDegeneratePath (its computePathCenterDistance and computeOutsideDistance have a separate squared-style bug, LFV-741). The finder's style matrix found no mismatch in GeoStandardPath's computeNearestDistance, computePathCenterDistance and computeDeltaDistance for ordinary points on 2- and 3-point paths; that is not exhaustive (by reading, their degenerate branches for a point at the pole of a segment's plane skip toAggregationForm, GeoStandardPath.java:1592-1594, :1687-1689; not executed, not part of this bug).
• Likelihood: plausible. No caller was found in Lucene, nor by a read-only code search of Elasticsearch, OpenSearch and Solr (three downstream queries were rate-limited), and no report was found since 9.5.0.
• Related, separate bugs (other root causes; the fix here does not fix them): LFV-741 (GeoDegeneratePath's computePathCenterDistance and computeOutsideDistance with the squared styles, same 2022 refactor), LFV-742 (path distance methods throw "Can't find world intersection" at the edge of a cutoff plane's tolerance band), LFV-735 (outside distance measured to the end point instead of the cap edge), LFV-736 (WGS84 end caps shifted toward the equator).
• Workaround: ask for LINEAR (or NORMAL) and square it yourself.

The test builds the path with GeoPathFactory.makeGeoPath, computes LINEAR at the point (0.05, 0.25) and asserts that LINEAR_SQUARED equals its square within 1e-12. It sits in org.apache.lucene.spatial3d.geom next to TestGeoPath by convention; it needs no package access. Seed DEADBEEF fails with assertions on and off, and 10 of 10 random seeds fail with the same message. It checks LINEAR_SQUARED only, which is representative: both squared styles go through the same line, and with the fix the finder's test, which also checks NORMAL_SQUARED at this point, passes.

The same file fails unchanged on the 10.6.0, 10.5.2, 9.12.3 and 9.5.0 jars and passes on 9.4.2, the last release before the introducing commit. As a regression test, use this same test, as its own class or as one method in TestGeoPath; :lucene:spatial3d:check -x test passes with it.

TestGeoPathSquaredDistance > testLinearSquaredIsSquareOfLinearInsideSegment FAILED
    java.lang.AssertionError: LINEAR_SQUARED vs LINEAR^2 (LINEAR=0.29934425859988006) expected:<0.08960698515671187> but was:<0.00802941178887518>
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:60C682F5EE012AE8]:0)
        at junit@4.13.2/org.junit.Assert.fail(Assert.java:89)
        at junit@4.13.2/org.junit.Assert.failNotEquals(Assert.java:835)
        at junit@4.13.2/org.junit.Assert.assertEquals(Assert.java:555)
        at org.apache.lucene.spatial3d.geom.TestGeoPathSquaredDistance.testLinearSquaredIsSquareOfLinearInsideSegment(TestGeoPathSquaredDistance.java:31)

# 0.08960698515671187^2 = 0.00802941178887518; -Ptests.asserts=false: the same failure

• The contract says convert back once. DistanceStyle javadoc, on all three methods: "Distances should be converted to aggregation form before aggregation is attempted, and they should be converted back from aggregation form to yield a final result." (DistanceStyle.java:109-111, :122-124, :139-141).
• The aggregation forms exist for exactly this case. LUCENE-7942 (#8991, Karl Wright, 2017), "For Geo3d paths, aggregating distance values using "+" is not adequate for squared distances": "The xxxSquaredDistance computations, for example, do not produce true squared distances ... I propose adding support in Distance for aggregation, which would allow distance calculators to compute an accurate distance". CHANGES.txt:7864-7865 (7.1.0): "Distance computations now have the ability to accurately aggregate distances, rather than just doing sums."
• The same object and its sibling give d^2. Points only in a cap of the same path get LINEAR^2. GeoDegeneratePath, which the factory returns for width 0, converts back once per return in computeDistance (GeoDegeneratePath.java:223-253) and gives LINEAR^2 on its line where the width-0.1 path gives LINEAR^4 (executed); its other two path-distance methods have their own squared-style bug (LFV-741).
• It used to work. 9.4.2 returns LINEAR^2 for the repro point, and so do both parent commits of the introducing change (executed).
• History says leftover. c9c27c755a4 ("Make sure use of aggregation form is consistent throughout, ...") added the root conversion and removed the cap's, but left the segment's. The next commit, 1ded41ea20d, removed the same leftover from PathSegment.nearestDistance, not from PathSegment.distance. That it was an oversight is an inference; nobody said so.
• No test covers it. No Lucene test uses a squared style; TestGeoPath measures path distances with ARC only.

The strongest case for "intended" (built by the refute lens, the verifiers and the adjudication): the geom distance API is @lucene.experimental, GeoStandardPath is @lucene.internal (GeoStandardPath.java:34), GeoDistance calls the value an "internal distance" whose implementations "should clarify how this is computed when it's non-obvious" (GeoDistance.java:33-34, :46-47), and no sentence literally promises LINEAR_SQUARED(path) == LINEAR(path)^2.

Why it fails: experimental is a back-compat caveat, not a licence for wrong values, and GeoPathFactory is public and returns this class for any width of at least about 3.14e-12 rad; DistanceStyle prescribes converting back once, and LUCENE-7942 exists to make squared path distances true squared distances; whatever "internal distance" means, one path cannot return d^2 in its caps and d^4 in its segments, with a 6.3x jump between them and an order that disagrees with LINEAR. Verdict: not intended, a defect (the independent reproduction, the intended-lens cross-model review, the intended-behaviour check and the adjudication; the adjudication with high confidence).

One fix was built and run through the whole spatial3d module; the alternative was not built.

In GeoStandardPath.java:1547-1548, return distanceStyle.aggregateDistances(startingDistance, pathDistance) without fromAggregationForm. This matches what c9c27c755a4's diff did for the caps: every leaf returns aggregation form, like BaseSegmentEndpoint.distance (:885), PathNode.distance compares like with like, and the root converts once (:306). Comparing aggregation forms in Math.min is safe because the conversions are monotonic on non-negative values; with the fix, points exactly on a cutoff plane get LINEAR^2 too (executed).

       final double pathDistance = pathDistance(distanceStyle, x, y, z);
-      return distanceStyle.fromAggregationForm(
-          distanceStyle.aggregateDistances(startingDistance, pathDistance));
+      return distanceStyle.aggregateDistances(startingDistance, pathDistance);
     }

Trade-offs

None observed in the validation below. ARC, LINEAR and NORMAL cannot change (identity conversions), so path queries and sorts are untouched; with the same change the path-query and path-sort fingerprint was identical (executed).

What the checks validated

• Gradle, scratch worktree of 645d4fe14a20, OpenJDK 25.0.3, seed DEADBEEF, 2026-10-31: stock, the repro fails; with the fix it passes, assertions on and off.
• The whole :lucene:spatial3d:test (20 suites, including TestGeoPath 11 of 11 and the repro): 182 tests, 3 skipped, 0 failures.
• :lucene:spatial3d:check -x test (checkGoogleJavaFormat, ecjLintMain, ecjLintTest, forbiddenApisMain, forbiddenApisTest): passes.
• Earlier, the same change made the independent reproducer's five tests pass on main ea57360912ed, and the finder's test pass at the introducing commit c9c27c755a4.

Make BaseSegmentEndpoint.distance convert back too and drop the fromAggregationForm at :306.

Trade-offs

Same distance() values, but it touches two places and reverses the structure c9c27c755a4 introduced, which converts once at the root in every path-distance method (:278-333); the commit message itself says only that aggregation-form use should be "consistent throughout". Not recommended.

Method: non-formal (code reading; hunt unit for spatial3d geometry).

• While auditing the GeoStandardPath component tree for the unit's metamorphic harness, the finder, an AI tool, traced the aggregation-form flow through PathSegment.distance, BaseSegmentEndpoint.distance, PathNode.distance and GeoStandardPath.distance.
• A style-consistency matrix on the stock jar compared each squared style with the square of its plain style for four distance methods on 2- and 3-point paths: only computeDistance disagreed, and only for segment points.
• A public-API JUnit confirmed it; an independent reproducer then wrote its own test from the claim alone, with an analytic oracle and a random sweep. The one-test repro was cut from the finder's test on 2026-10-29.