All findings · Lucene finding · LFV-678 · geo3d-linestring-buffer-degrees-as-radians

Geo3d spatial context: a line string's buffer is read as radians, so a 1-degree buffer is 57 degrees wide

In the Geo3d spatial context, lineString().buffer(1) builds a corridor reaching 57.3 degrees from the line instead of 1 degree, because the buffer goes to spatial3d as radians without the degrees-to-radians conversion the same factory applies to circles. Buffers above about 1.57 throw Cutoff angle out of bounds.

Summary

Priority
Low, rank 350Needs buffered line strings in the opt-in experimental Geo3d context, where the error is conspicuous (about 57 times too wide, or an exception).
Impact
silent wrong results
Since
7.1.0 (2017), LUCENE-7951, the change that added the Geo3d spatial contextintroduced 2017-09-20 · about 9 years
Affected
every release from 7.1.0 to 10.5.2 (all 72 run), 10.6.0 (by reading) and main
Status
documented
Found by
code reading, then a differential check against spatial4j's own geo context (non-formal)

AI-assisted page: this finding was first signalled by code reading or testing and then checked with AI tools; the checks it went through, and what each found, are under Prior reports, history and checks. Not yet triaged by a human. Snapshot of 2026-11-06.

The finding in 30 seconds

spatial4j's geo conventions put distances in degrees, and the Geo3d shape factory converts degrees to radians for circles and rectangle buffers. Its line-string builder hands the buffer to spatial3d unconverted, so buffer(1) means 1 radian: a corridor 57.3 times too wide. Buffers above about 1.57 are rejected.

Map from -60 to 60 degrees: the line (0,0) to (10,0) with the expected 1-degree corridor, a thin band along the line, and the actual corridor, a rounded shape reaching 57.3 degrees on every side that contains the points (5, 20) and (5, -40). Beside it: the bounding box, the 3 index hits where 1 is right, the rejected buffers above 1.5708 and the one-line fix.
The 1-degree corridor you ask for (orange, along the line) and the corridor the Geo3d context builds (blue, traced with relate on main): it reaches 57.3 degrees and contains (5, 20) and (5, -40). Select the still to open it at full size. Generated with AI, checked against the code.
  1. You make a spatial context with Geo3dSpatialContextFactory and build the line (0,0) to (10,0) with buffer(1). In a geographic spatial4j context distances are in degrees, so you ask for everything within 1 degree (about 111 km) of the line.
  2. You expect the point (5, 20), 20 degrees north of the line, to be outside: spatial4j's own SpatialContext.GEO and this factory's circle(5, 0, 1) both say DISJOINT.
  3. You get CONTAINS. The corridor's bounding box is [-57.3, 67.3] x [-57.3, 57.3]: the buffer was used as 1 radian, 57.3 degrees. A spatial index query with this line returns 3 of 4 points where 1 is right. No error.
  4. Why: Geo3dLineStringBuilder.build() hands the buffer straight to GeoPathFactory.makeGeoPath, whose width is an angle in radians; circle() and rectangle buffers convert degrees first. Reading spatial4j's polyshape string of the line gives the same 57.3-degree corridor.
  5. Any buffer above about 1.5708 throws IllegalArgumentException: Cutoff angle out of bounds, so a 2-degree corridor cannot be built. Every release from 7.1.0 to 10.6.0 (10.6.0 by reading) and main are affected; the fix is one multiplication.

Who is affected

Applications that use the opt-in Geo3d spatial context with buffered line strings, as indexed shapes or as query shapes, through the builder, the polyshape format or GeoJSON's buffer. Every such line is about 57 times too wide, or rejected above about 1.57; unbuffered lines, circles, rectangles and polygons are not affected by this mismatch. Index size does not matter.

Mechanism, terms and numbers672 words

spatial4j's documented geographic conventions put distances in degrees (LineStringBuilder.buffer itself names no unit, so its unit is inferred from those conventions and spatial4j's readers). The spatial-extras Geo3dShapeFactory, the shape factory of the opt-in, experimental Geo3dSpatialContextFactory, converts degrees to radians at its boundary for circles, 2-D points and rectangle buffers, but its line-string builder does not: buffer(distance) stores the value as given and build() passes it to spatial3d's GeoPathFactory.makeGeoPath, whose width "is the width of the path, measured as an angle", in radians. So the line (0,0) to (10,0) with buffer(1) reaches 57.3 degrees from the line: bounding box [-57.3, 67.3] x [-57.3, 57.3] instead of [-1, 11] x [-1, 1], and it contains the point (5, 20), which spatial4j's own geo context and the same factory's 1-degree circle leave outside. The same happens when the Geo3d context reads spatial4j's polyshape string of that line. Any buffer above about 1.5708 throws IllegalArgumentException: Cutoff angle out of bounds. In an index, an Intersects query with the 1-degree line returned 3 of 4 points where 1 is right. The code dates from LUCENE-7951; every release from 7.1.0 (2017) to 10.6.0 (10.6.0 by reading) and main are affected, and the fix is one multiplication in build().

Terms

  • Spatial context: spatial4j's SpatialContext fixes the units and builds shapes through its ShapeFactory. In a geographic context x is longitude, y is latitude, and distances are in degrees (1 degree is about 111 km).
  • Geo3d spatial context: Geo3dSpatialContextFactory in lucene-spatial-extras (@lucene.experimental, opt-in; default planet model: the sphere) makes a context whose Geo3dShapeFactory builds lucene-spatial3d shapes. spatial3d works in radians, so the factory converts at its boundary.
  • Buffered line string: lineString().pointXY(...).buffer(d).build(), the corridor of all points within d of the line. In the Geo3d context it becomes a spatial3d GeoStandardPath.

The bug

  1. Geo3dLineStringBuilder.buffer(distance) stores the value unchanged: this.distance = distance; (Geo3dShapeFactory.java:308-312).
  2. build() calls GeoPathFactory.makeGeoPath(planetModel, distance, points.toArray(GeoPoint[]::new)) (:314-319).
  3. makeGeoPath documents its width as "the width of the path, measured as an angle" (GeoPathFactory.java:34), a spatial3d angle in radians, and builds a GeoStandardPath. That constructor rejects widths outside (0, pi/2] with "Cutoff angle out of bounds" (GeoStandardPath.java:80-81).
  4. Elsewhere the factory converts degrees for circles, 2-D points and rectangle buffers: circle() passes distance * DistanceUtils.DEGREES_TO_RADIANS (Geo3dShapeFactory.java:175-197), pointXY converts coordinates (:278-286; pointXYZ has a different boundary bug, LFV-679), and Geo3dRectangleShape.getBuffered converts its distance (Geo3dRectangleShape.java:169-173). The context's distance calculator and circle radii report degrees. normDist is the identity (:114-117).
  5. spatial4j's readers feed the same builder: the polyshape reader passes the same normDist(arg) to a circle's radius and to a line's buffer, and the GeoJSON reader converts a km buffer to degrees before calling buffer(). So one polyshape document gives a 1-degree circle but a 1-radian line. The deprecated lineString(List, double) and the multi-line builder use the same builder too (both executed: the 57.3-degree corridor, CONTAINS (5, 20)). spatial4j registers the GeoJSON reader only when Noggit's org.noggit.JSONParser is on the classpath; it was not on the tested classpaths, so that path was read, not run.

The numbers

  • Line (0,0) to (10,0), buffer(1), default sphere: bounding box [-57.3, 67.3] x [-57.3, 57.3] (spatial4j: [-1, 11] x [-1, 1]). Along x = 5 the edge lies between y = 57.0 (inside) and 57.5 (outside): 1 radian is 57.2958 degrees.
  • The same line with buffer(Math.toRadians(1)) is the 1-degree corridor (bounding box [-1, 11] x [-1, 1]).
  • buffer(1.5708) (just above pi/2), buffer(1.6), buffer(2) and buffer(90.01): IllegalArgumentException: Cutoff angle out of bounds; buffer(1.5707) builds. Exactly Math.PI / 2 passes that check and, for the probe's two-point line, fails later with "path too wide". The bounding box of buffer(1.5) spans all longitudes.
  • WGS84 behaves the same: a point 2.98 degrees from the 1-degree line is inside.
  • Not affected: unbuffered lines (a zero width becomes a degenerate path), circles, rectangles, polygons, and spatial4j's default geo context. WKT BUFFER(LINESTRING(...), d) is not supported in the Geo3d context at all.
Exposure and likelihood322 words
  • Who is hit. Users of Geo3dSpatialContextFactory (any planet model) who build buffered line strings: lineString().buffer(d), the deprecated lineString(List, double), multi-line builders (all executed), the polyshape format (executed) or the GeoJSON "buffer" extension when the optional Noggit JSON parser is on the classpath (by code reading). As document shapes or as query shapes ("documents within d of this route"); RecursivePrefixTreeStrategy with Geohash or S2 trees and SerializedDVStrategy all see the wide shape.
  • Default vs opt-in. Opt-in twice: the Geo3d context is not the default, and only buffered lines are wrong. WKT, one of spatial4j's default input formats, cannot express a buffered line in this context.
  • Effect. Buffers below about 1.57: a corridor 57.2958 times wider than requested, with no error. In an index of 4 points an Intersects query with the 1-degree line returned 3 where 1 is right; in a 600-document index (514 live) it matched 359 documents on Geohash and S2 prefix trees and 354 on SerializedDVStrategy, where the fix matches 0 (a synthetic uniform sample of points: the counts show that every strategy sees the wide shape, not how often real queries over-match). Indexed buffered lines can give false positives for intersection queries that touch the extra area. Buffers above about 1.57 (callers mean about 175 km) throw IllegalArgumentException while the shape is built; a 200 km GeoJSON buffer (about 1.8 degrees) is in that range.
  • Still honoured. Other filter clauses and deletions: 0 documents from another tenant and 0 deleted documents were returned.
  • Lasting. SerializedDVStrategy stores the path with its angle and prefix-tree terms encode the wide corridor, so documents indexed while the bug is present stay wide after a fix until they are reindexed (code reading, not executed).
  • How likely. Rare: the context is opt-in and buffered lines need the builder, polyshape or GeoJSON. For those callers every buffered line is wrong, and conspicuously so; no report was found in the nine years since 7.1.0.

Minimal repro

The test shows that in the Geo3d spatial context a line string's buffer(1) is used as 1 radian (57.3 degrees) instead of 1 degree.

TestGeo3dLineStringBuffer.java (JUnit test)32 lines
/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.apache.lucene.spatial.spatial4j;

import org.apache.lucene.tests.util.LuceneTestCase;
import org.locationtech.spatial4j.shape.ShapeFactory;
import org.locationtech.spatial4j.shape.SpatialRelation;

public class TestGeo3dLineStringBuffer extends LuceneTestCase {

  public void testBufferIsInDegrees() {
    ShapeFactory f = new Geo3dSpatialContextFactory().newSpatialContext().getShapeFactory();
    // expected: a line buffered by 1 degree is DISJOINT from a point 20 degrees away; actual: CONTAINS
    assertEquals(
        SpatialRelation.DISJOINT,
        f.lineString().pointXY(0, 0).pointXY(10, 0).buffer(1).build().relate(f.pointXY(5, 20)));
  }
}
Command
# in a copy of this finding's repro folder (repro/LFV-678/, which holds src/)
# Lucene checkout at the tested commit, built with: ./gradlew --no-daemon assemble   (JDK 25)
LUCENE=/path/to/lucene; V=12.0.0-SNAPSHOT
CP=$LUCENE/lucene/core/build/libs/lucene-core-$V.jar
CP=$CP:$LUCENE/lucene/codecs/build/libs/lucene-codecs-$V.jar
CP=$CP:$LUCENE/lucene/test-framework/build/libs/lucene-test-framework-$V.jar
CP=$CP:$LUCENE/lucene/spatial-extras/build/libs/lucene-spatial-extras-$V.jar
CP=$CP:$LUCENE/lucene/spatial3d/build/libs/lucene-spatial3d-$V.jar
CP=$CP:randomizedtesting-runner-2.9.2.jar:junit-4.13.2.jar:hamcrest-3.0.jar   # test-framework dependencies (Maven Central)
CP=$CP:junit-platform-console-standalone-6.1.3.jar   # JUnit Platform console launcher (Maven Central)

javac -d classes -cp "$CP" $(find src -name '*.java')

# run 1: assertions on, seed DEADBEEF
java -ea -esa --add-modules jdk.incubator.vector --enable-native-access=ALL-UNNAMED -Dtests.seed=DEADBEEF \
     -cp "classes:$CP" org.junit.platform.console.ConsoleLauncher \
     execute --select-class org.apache.lucene.spatial.spatial4j.TestGeo3dLineStringBuffer
# the module(s) spatial-extras also need their third-party runtime jars
Expected vs actual
Expected lineString (0,0)-(10,0) with buffer(1) to be DISJOINT from point (5,20); actual: CONTAINS (expected:<DISJOINT> but was:<CONTAINS>), with -ea and -da.

All files of this repro: repro/LFV-678/

More files, environment, full output34 words

Environment

luceneLucene main e1faeba08bde (2026-11-02)
jdk25.0.3+9-LTS

Observed output

# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
    => java.lang.AssertionError: expected:<DISJOINT> but was:<CONTAINS>
       org.junit.Assert.fail(Assert.java:89)
       org.junit.Assert.failNotEquals(Assert.java:835)
       org.junit.Assert.assertEquals(Assert.java:120)
       org.junit.Assert.assertEquals(Assert.java:146)
       org.apache.lucene.spatial.spatial4j.TestGeo3dLineStringBuffer.testBufferIsInDegrees(TestGeo3dLineStringBuffer.java:28)

Why it looks unintended

  • spatial4j: circle distances are in the units of x and y, and its BUFFER argument "is the buffer distance in degrees".
  • The polyshape reader passes one value to circle radii and line buffers: a 1-degree circle, a 1-radian line.
  • The one radian caller, testGeoPathRect, was ported from a direct spatial3d call; its checks ignore the unit.
The full argument242 words
  • Contract (inferred from spatial4j, not quoted from buffer()). spatial4j ShapeFactory.circle: "The units of "distance" should be the same as x & y." lineString(List<Point> points, double buf): "with a buffer distance along the line in all directions". LineStringBuilder.buffer(double) itself has no javadoc. spatial4j's FORMATS.md: "The BUFFER keyword ... The 2nd argument is the buffer distance in degrees."
  • Lucene's description of the bridge. CHANGES: "LUCENE-7951: Spatial-extras has much better Geo3d support by implementing Spatial4j abstractions: SpatialContextFactory, ShapeFactory, BinaryCodec, DistanceCalculator." The package: "Spatial4j stuff that ideally belongs in Spatial4j (isn't related to Lucene)." No javadoc, CHANGES or MIGRATE text says line buffers are radians.
  • The factory itself. Circles, 2-D points, rectangles, rectangle buffers, the distance calculator and circle radii all use degrees at the boundary. Only the line buffer does not. Its own javadoc names the unit for circles: setCircleAccuracy sets "the accuracy for circles in decimal degrees" (Geo3dShapeFactory.java:82-85).
  • The readers. The polyshape reader passes the same normDist(arg) to circles and line buffers, and normDist is the identity here; GeoJSON converts km to degrees before buffer(). A radians-only builder is wrong for every parsed input, whatever is documented.
  • Maintainer remarks. In the LUCENE-8086 review, dsmiley asked whether the factory's circleAccuracy means "decimal degrees as is used in other parts of the Spatial4j API"; in the LUCENE-7951 review they wrote that verification can "assume decimal degrees here (-/+ 180 etc.) since Geo3d assumes this". Nobody discussed the line buffer's unit.
The devil's-advocate check200 words

Strongest case for "intended" (built by a devil's advocate; the reviews from another model family, OpenAI GPT-5.6, rated it moderate and strong): LineStringBuilder.buffer states no unit; spatial3d is radians-native; the bridge is @lucene.experimental; the behaviour has not changed since 2017; and the only caller of the builder's buffer() outside the factory, ShapeRectRelationTestCase.testGeoPathRect, passes (random().nextInt(89) + 1) * DEGREES_TO_RADIANS, a range that lies within GeoStandardPath's (0, pi/2] bound (1 to 89 degrees, at most 1.553 rad).

Why it fails: the radian width is older than the builder. It dates from LUCENE-6196 (2015), when the test built spatial3d's new GeoPath(width) directly, before the Geo3d spatial context existed; in 7.2.0 the test still called GeoPathFactory.makeGeoPath(planetModel, width, points) directly, where radians are right; LUCENE-8086 (2017-12-13, first in 7.3.0) moved it onto the builder and kept the radian width. The same test passes its circle radius in degrees, and it checks relation consistency, which does not depend on the width's unit. spatial4j's readers can only ever pass degrees. @lucene.experimental allows API changes, not wrong results, and nine quiet years speak to compatibility, not intent. Every verdict calls it a defect; what remains is the compatibility cost for callers who already pass radians.

Suggested fix

Recommended A: convert in build(), and switch testGeoPathRect to degrees

In Geo3dLineStringBuilder.build(), pass distance * DistanceUtils.DEGREES_TO_RADIANS to GeoPathFactory.makeGeoPath, and switch testGeoPathRect to degree widths so its corridors stay as wide as today. The new test passes, and so do the whole spatial-extras module (120 tests, 0 failures, 5 skipped) and the module checks. Callers who passed radians get narrower lines; stored buffered lines need reindexing.

// Geo3dShapeFactory.Geo3dLineStringBuilder.build()
GeoPath path =
    GeoPathFactory.makeGeoPath(
        planetModel,
        distance * DistanceUtils.DEGREES_TO_RADIANS,
        points.toArray(GeoPoint[]::new));

// ShapeRectRelationTestCase.testGeoPathRect
final double width = random().nextInt(89) + 1; // degrees, like the circle radius above
All options and trade-offs4 options ยท 614 words

Built and validated in a scratch worktree at a9779ef3c193 with gradle, and with the standalone runner against the finder's, the independent reproducer's, the devil's advocate's and the cross-model reviewer's tests. Any option changes the unit for callers who already pass radians, so it needs a CHANGES entry that says so.

A: convert in build(), and switch testGeoPathRect to degrees (validated by the checks) (recommended)

One multiplication in Geo3dLineStringBuilder.build(), the conversion circle() already does. In ShapeRectRelationTestCase.testGeoPathRect, pass the width in degrees (and drop the now unused DEGREES_TO_RADIANS static import), so the test keeps building the same 1-89 degree corridors as today. Add this page's test as the regression test. Patch: +116/-4 in 3 files (5 lines in the factory, 4 in the old test, the new 111-line test).

// Geo3dShapeFactory.Geo3dLineStringBuilder.build()
GeoPath path =
    GeoPathFactory.makeGeoPath(
        planetModel,
        distance * DistanceUtils.DEGREES_TO_RADIANS,
        points.toArray(GeoPoint[]::new));

// ShapeRectRelationTestCase.testGeoPathRect
final double width = random().nextInt(89) + 1; // degrees, like the circle radius above

Trade-offs

  • Behaviour change: callers who already pass radians, as testGeoPathRect does today, get lines 57.3 times narrower. A CHANGES entry should say so (for example under "Changes in Runtime Behavior").
  • Documents indexed with buffered lines keep the wide shape until they are reindexed (code reading, not executed).
  • Without the test update testGeoPathRect still passes (run), but it would build corridors of 0.017 to 1.55 degrees instead of 1 to 89 and stop covering wide paths.
  • Cost: one multiplication per line built.

What the checks validated

  • Gradle, scratch worktree at a9779ef3c193, seed DEADBEEF: with the factory fix alone, the new test 3/3 with assertions on and off and the whole spatial-extras module 120 tests, 0 failures, 5 skipped (4 @Nightly, 1 @Ignore: TestBBoxStrategy.testBasicOperations); with the fix and the test update, the module again 120 tests, 0 failures, the same 5 skipped, the rect-relation suites with two more seeds (assertions on and off) and with -Ptests.nightly=true (15 tests, including testGeoPathRect on the sphere and on the ellipsoidal class, which picks WGS84 or CLARKE_1866 at random, 0 failures), and :lucene:spatial-extras:check -x test (google-java-format, ecjLint, forbiddenApis) passes. The nightly rect-relation tests also pass with the fix alone and on stock. The fourth @Nightly test, TestGeo3dRpt, passes with the full patch (4 tests, run by the fact check), so only the @Ignore test never ran.
  • Standalone runner, patched class shadowed: this page's test 3/3 (assertions on and off), the finder's 2/2, the independent reproducer's 5/5, the devil's advocate's 5/5, and the index check 2/2 (1 hit instead of 3).

B: convert in buffer() instead (validated by the checks)

Store the converted value: this.distance = distance * DistanceUtils.DEGREES_TO_RADIANS;. Same behaviour as A; a matter of taste (A keeps the field in the caller's unit).

Trade-offs

Same compatibility and reindexing notes as A; it also needs the testGeoPathRect update.

What the checks validated

Standalone runner only, patched class shadowed: this page's test 3/3 with assertions on and off.

Optional add-on: reject buffers above 90 degrees with a message in degrees

After A, a buffer above 90 degrees (90.01 in the probe) still fails with "Cutoff angle out of bounds", which names a spatial3d parameter the caller never passed. A check in build() with the limit in degrees would be clearer.

Trade-offs

Not written or tested. For the probe's two-point equatorial line, a buffer of exactly 90 degrees already fails later with "path too wide"; other lines were not tried.

Document radians instead (not recommended)

Keep the behaviour and state on Geo3dShapeFactory that line buffers are radians.

Trade-offs

Leaves every parsed buffered line wrong: spatial4j's polyshape and GeoJSON readers pass degrees, and normDist cannot convert for lines alone, because the polyshape reader uses the same normDist for circle radii, which circle() already converts.

Downstream note

An application that used a buffered-line query as its visibility boundary (a geofence) would admit documents up to 57 degrees from the line instead of 1, while other filter clauses and deletions are still honoured.

How it was found

Non-formal: while writing a differential harness that compares the Geo3d bridge with spatial4j's own geo context, the finder read the unit conversion of every Geo3dShapeFactory method and saw that the line-string buffer has none. A differential check then showed the same 1-degree line DISJOINT from a point 20 degrees away in spatial4j and CONTAINS in the Geo3d context.

The method in full128 words

Method: code reading plus differential testing (the non-formal tier), in a hunt unit on the spatial-extras bridge between spatial4j and spatial3d.

  • The unit's harness compares shapes built by the Geo3d context with the same shapes built by spatial4j's SpatialContext.GEO and by the same factory's circles. While writing its line generator the finder checked each factory method's degrees-to-radians handling and found none for line buffers; the harness now passes buffers in radians so that its other checks do not depend on this question.
  • A differential probe and then a JUnit confirmed it on the stock jars: the 1-degree line contains (5, 20) in the Geo3d context and not in spatial4j's, also when read from spatial4j's polyshape string.
  • A later verifier found the second symptom: buffers above about 1.57 throw.

Prior reports, history and checks

Prior reports

No prior report: two automated runs (138 queries) and 42 manual searches on 2026-10-27 over GitHub issues, PRs and commits, the changed-file lists of the 409 open PRs, JIRA, the Lucene and Solr lists and the web. Closest: #9000, which added the builder, and #9134, which moved a test onto it with radian widths; neither discusses the unit.

Searches5 searches

Nothing by anyone reports or discusses the unit of the Geo3d line buffer or the "Cutoff angle out of bounds" failure. The closest items are below.

What was searched, when, and what turned up
WhereWhenHowFound
GitHub issues, PRs and commits (apache/lucene), automated2026-10-27two prior-report tool runs, 45 and 93 queries (class names, error text, buffer/radians/degrees phrasings); 2,966 distinct hits in the second, the positive control #9000 at rank 1no report; related: #9000, #9134, #9219, #9537, #9294, #11956
GitHub, manual2026-10-2742 searches over apache/lucene, apache/lucene-solr, apache/solr, Elasticsearch, OpenSearch, spatial4j and all of GitHubnothing matching; spatial4j #198 shows a user passing a degree value to lineString().buffer()
Open PRs and upstream branches2026-10-27changed-file lists of all 409 open apache/lucene PRs (4 capped lists re-read through the REST API; #16911, a 7,630-file branch decommission, listable only to 3,000 files); commits by path on main, branch_10x, branch_10_6, branch_10_5, branch_9xnone touches the fix site; upstream main still passes the raw distance
JIRA and mailing lists2026-10-2712 JQL queries (LUCENE, SOLR); 16 list queries over dev@, java-user@, issues@, solr-user@, users@solr, dev@solrLUCENE-9263 (a different unit bug, in Geo3DPoint); SOLR-11382 (the Solr integration, no buffers)
Web2026-10-279 queries (one more throttled)the Solr reference guide says distances in WKT buffers "are still in degrees"; no report

History

Introduced in 7.1.0 by LUCENE-7951 (035523ba7a86 on main, 85491401d54c on branch_7x, 2017-09-20), the change that added the Geo3d spatial context; not a regression. The finder's test fails on all 72 releases from 7.1.0 to 10.5.2 (assertions on) and on main; never fixed.

Versions tested6 rows

Introduced by 035523ba7a86 (main) and 85491401d54c (branch_7x), 2017-09-20, "LUCENE-7951: Spatial4j implementations using Geo3d" (#9000). The code matches lucene-solr#246 by iverase (closed unmerged); dsmiley committed it. Later commits on these lines only reformat (2cbf261032d, LUCENE-9570) or modernise toArray (dbf0d1426f1, #16056). LUCENE-8086 (d66d9549d7c, 2017-12-13, first in 7.3.0) moved testGeoPathRect onto the builder and kept its radian width, which dates from LUCENE-6196 (2015), when the test built spatial3d's GeoPath directly. Upstream main a95c754ee003, branch_10x and branch_10_6 are unchanged.

Versions and branches executed, with results
Version or commitVerdictResult
main a9779ef3c193 affectedthis page's test fails 3 of 3 (gradle and standalone, assertions on and off)
10.5.2 affectedthis page's test fails 3 of 3 (assertions on and off)
all 72 releases from 7.1.0 to 10.5.2 affectedthe finder's test fails 2 of 2 with identical messages (assertions on; spatial4j 0.6, 0.7 and 0.8)
7.1.0 jars, factory shadowed commit checkunchanged 7.1.0 source fails; with only the conversion added it passes
7.0.1 no APIno Geo3d spatial context; the test does not compile
fix on main not affectedthis page's test, the whole spatial-extras module (120 tests, 0 failures, 5 skipped) and the @Nightly tests pass

Independent checks

Reproduced on main by the finder, an independent reproducer, two executed reviews by another model family and a devil's advocate, each with its own test, and re-run by the adjudication and the follow-up. All rated it real; the case for intended behaviour (the only caller outside the factory passes radians) was rated moderate to strong but outweighed.

Checks and verdicts318 words

Each check ran its own tests on main a9779ef3c193 (the cross-model reviews on ea57360912ed, same sources, re-run on a9779ef3c193).

Each independent check and its verdict
CheckVerdictWhat it found
finder candidateTwo-test JUnit fails on main (assertions on and off) and 7.1.0, 7.7.3, 8.11.4, 9.12.3, 10.5.2; the one-line fix passes; the full spatial-extras suite with all six fixes of the hunt unit: 117 tests, 0 failures, as on the baseline (clean checkout of ea57360912ed, same spatial sources).
intended review (OpenAI GPT-5.6, executed) real, high; intended argument moderateOwn contract test fails 2 of 2 on stock (builder and polyshape) and passes with the fix; the fix passes the runnable upstream TestGeo3d.
Independent reproducer realOwn 5-test JUnit with the context's own distance calculator as oracle fails 5 of 5 on main, 10.5.2 and 9.12.3, on the sphere and WGS84; passes with the fix.
Refute review (OpenAI GPT-5.6, executed) real, high; intended argument strong but failingOwn test (another line, buffer 0.5, a point 15 degrees away) fails only the width check; buffer(2.0) throws; the index check returns 3 hits instead of 1.
Devil's advocate real, highBuilt the strongest intended case and showed why it fails; own test fails its 3 buffer checks on main and 10.5.2 while its 2 unit controls pass; found the 'Cutoff angle out of bounds' symptom.
adjudication confirmed, highRe-ran the independent reproduction's test (5 of 5 fail, assertions on and off; pass with the fix) and the finder's; security none; not a duplicate.
Follow-up keepHistory on all 72 releases and the 7.1.0 commit check; prior-art search: new; security probe: none.
this page reproduced; fix validatedNew 3-test JUnit fails on main and 10.5.2 (gradle and standalone, assertions on and off); the one-line fix plus the test update passes the whole spatial-extras module, its @Nightly rect-relation tests and its checks.

Diagram file

The diagram as a still image, generated with AI and checked against the code. It is under 10 MB, the limit for a GitHub attachment.

Sources and provenance

Finding record, Lucene and upstream sources, repro files and diagram notes1,334 words

Finding record

LFV-678, snapshot of 2026-11-06

Registry title
Geo3dShapeFactory passes a line string's buffer distance to spatial3d's GeoPathFactory, which takes radians, without the degrees-to-radians conversion its circle() applies, so in the Geo3d spatial context a line buffered by 1 degree is 1 radian (57.3 degrees) wide and buffers above about 1.57 throw
Area
spatial-extras / spatial4j bridge to spatial3d (Geo3dShapeFactory line-string builder, Geo3dLineStringBuilder.build), reached through the public LineStringBuilder API, the deprecated lineString(List, double), multi-line builders and the Geo3d context's polyshape reader (and the GeoJSON reader when its optional parser is present)
Priority
Low, rank 350: Needs buffered line strings in the opt-in experimental Geo3d context, where the error is conspicuous (about 57 times too wide, or an exception).
Impact class
silent wrong results
Impact note
Applications on the opt-in, @lucene.experimental Geo3d spatial context (Geo3dSpatialContextFactory) that index or query buffered line strings (builder API, deprecated lineString(List, double), multi-line builders, the polyshape reader; the GeoJSON reader when its optional parser is present, not executed) get a corridor about 57.3 times wider than requested with no error, so indexed corridors and corridor queries match far too much (an Intersects query on a RecursivePrefixTreeStrategy index returned 3 hits where 1 is right; in the security probe a 1-degree buffer matched 359 of 514 live points where the fix matches 0), and any buffer above about 1.5708 (meant as degrees) throws IllegalArgumentException 'Cutoff angle out of bounds'; unbuffered lines, circles and buffered rectangles are unaffected, ordinary scale, every release from 7.1.0 to 10.6.0 (10.6.0 by reading).
Summary
You use the Geo3d spatial context, build a line from (0,0) to (10,0) with buffer(1), and expect a corridor reaching 1 degree from the line, because spatial4j distances are in the units of x and y (degrees in a geo context) and spatial4j's standard geo context and the same factory's 1-degree circle both say a point at (5, 20) is DISJOINT. Instead the line CONTAINS that point and its bounding box is about [-57.3, 67.3] x [-57.3, 57.3]: Geo3dLineStringBuilder.build hands the buffer straight to GeoPathFactory.makeGeoPath, whose width is an angle in radians, while the factory's circle() and Geo3dRectangleShape.getBuffered convert degrees with DEGREES_TO_RADIANS. The same happens when the Geo3d context reads spatial4j's polyshape string for a 1-degree buffered line, and every buffer above about 1.5708 is rejected with 'Cutoff angle out of bounds', so a buffer of 2 degrees cannot be built at all.
Introduced
035523ba7a86773500c053129f65c9a6dc4fd384 (main) and its branch_7x cherry-pick 85491401d54c59e1593a5c9f13dca7bfcdda31a2 (the commit that shipped in 7.1.0): LUCENE-7951 'Spatial4j implementations using Geo3d' (issue apache/lucene#9000); the code matches apache/lucene-solr#246 by GitHub iverase (closed unmerged 2017-09-29), committed by GitHub dsmiley [read: git log -S 'makeGeoPath(planetModel, distance' and -S 'this.distance = distance' in the full-history clone return only this commit plus a reformat (2cbf261032d, LUCENE-9570) and a toArray modernisation (dbf0d1426f1, #16056), and no version ever documented a buffer unit; exec: on 7.1.0 jars, Geo3dShapeFactory shadowed unchanged fails 2 of 2 and with only the degrees-to-radians conversion in build() passes 2 of 2]. Not a regression: the defect came in together with the API. (2017-09-20 (both commits); first release 7.1.0 (release commit 2017-10-13, tag 2017-10-16))
First affected version
7.1.0 [exec: the finder's JUnit fails 2 of 2 (junitcore runner); 7.0.1 does not compile it because Geo3dSpatialContextFactory and Geo3dShapeFactory do not exist there, so the defect is as old as the API]
Affected versions
Every release from 7.1.0 to 10.5.2 (72 releases), 10.6.0 (by reading) and main a9779ef3c193; not fixed upstream. [exec, JDK 25, SHA-1-checked release jars] the finder's JUnit fails 2 of 2 on each of the 72 releases (-ea, seed DEADBEEF; 7.1.0-7.6.0 with the junitcore runner; 7.x to 9.0.0 with a copy that differs only in the LuceneTestCase import) and on main with -ea and -da; the independent reproduction's JUnit fails 5 of 5 on main (-ea, -da), 10.5.2 and 9.12.3; the intended-behaviour check's JUnit fails 3 of 5 (the 3 buffer assertions) on main (-ea, -da) and 10.5.2. [read] upstream main a95c754ee003 (2026-10-26), branch_10x and branch_10_6 still pass the raw distance; no open PR touches Geo3dShapeFactory.java, GeoPathFactory.java or GeoStandardPath.java. The 'Cutoff angle out of bounds' symptom is in every release by code identity (the same range check since 7.1.0; executed on main and 10.5.2 only).
Status
documented (brief, checks and a per-finding doc exist)
Verification
Confirmed by AI-run checks: an independent repro, a cross-model review that agreed, an intended-behaviour check, a prior-report search and a skeptical audit; not yet triaged by a person.
Found by
non-formal: code reading
Upstream and prior reports
none found (searched 2026-10-27, read-only; the prior-report record, verdict new): two scripted prior-report searches (45 queries with the web part pending, then 93 queries: 40 GitHub issue and PR queries on apache/lucene, 3 commit queries, 4 downstream-repository queries (Elasticsearch, OpenSearch, Solr), 32 queries over the dev, java-user, issues and commits lists, 6 JIRA queries and 8 web queries; 2,966 distinct hits, the positive control #9000 at rank 1, none about this defect), 42 manual GitHub search phrasings across apache/lucene, apache/lucene-solr, apache/solr, elastic/elasticsearch, opensearch-project/OpenSearch, locationtech/spatial4j and all of GitHub, the changed files of all 409 open apache/lucene PRs (none touches Geo3dShapeFactory.java, GeoPathFactory.java, GeoStandardPath.java or ShapeRectRelationTestCase.java), upstream commits by path on main, branch_10x, branch_10_5, branch_10_6 and branch_9x (upstream main a95c754ee003 still passes the raw distance), manual JIRA queries (LUCENE and SOLR) and the Lucene and Solr user and dev lists. Nothing reports or discusses the unit of Geo3dShapeFactory's line-string buffer or the 'Cutoff angle out of bounds' failure for larger buffers. Closest: the introducing issue LUCENE-7951 / #9000 and lucene-solr#246 ('The Spatial4j buffer extension is partially supported', which refers to the unsupported WKT BUFFER syntax; nobody discusses the unit of LineStringBuilder.buffer), LUCENE-8086 / #9134 with lucene-solr#288 (the change that ported testGeoPathRect onto the builder with radian widths, without discussing units; a review remark there says the Spatial4j API uses decimal degrees), and LUCENE-9263 (a different unit-conversion bug, in Geo3DPoint's meters-to-radians conversion). Gaps: GitHub search matches tokens, not code diffs; private downstream trackers cannot be searched; one web query was throttled.
Lucene commit tested
main a9779ef3c193 (commit of 2026-10-26; tested 2026-10-27) with gradle in a scratch worktree and with the standalone JUnit runner, assertions on and off; the 10.5.2 release jars (this page's test, assertions on and off). The finder's test ran on all 72 releases from 7.1.0 to 10.5.2 (assertions on). The fix was validated with gradle (this page's test, the whole spatial-extras module and its checks) and with the standalone runner against the finder's, the independent reproducer's, the devil's advocate's and the cross-model reviewer's tests. The cross-model reviews ran on ea57360912ed (same spatial-extras and spatial3d sources) and were re-run on a9779ef3c193.

Lucene and upstream sources

Finding record
  • lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dShapeFactory.java:50-55 (at a9779ef3c193) (@lucene.experimental, implements S2ShapeFactory), :114-117 (normDist is the identity), :175-197 (circle(x, y, distance) multiplies distance by DistanceUtils.DEGREES_TO_RADIANS), :204-213 (deprecated lineString(List, distance) goes through the same builder), :303-320 (Geo3dLineStringBuilder: buffer(distance) stores the value as given; build() calls GeoPathFactory.makeGeoPath(planetModel, distance, points) at :316-317)
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoPathFactory.java:30-44 (at a9779ef3c193) (maxCutoffAngle 'is the width of the path, measured as an angle'; below MINIMUM_ANGULAR_RESOLUTION a GeoDegeneratePath, else a GeoStandardPath)
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoStandardPath.java:76-86 (at a9779ef3c193) (IllegalArgumentException 'Cutoff angle out of bounds' unless 0 < maxCutoffAngle <= pi/2), :1457-1458 ('path too wide')
  • lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dRectangleShape.java:169-173 (at a9779ef3c193) (getBuffered converts distance with DEGREES_TO_RADIANS)
  • lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dSpatialContextFactory.java:28 (at a9779ef3c193) (@lucene.experimental)
  • lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/ShapeRectRelationTestCase.java:159-175 (at a9779ef3c193) (testGeoPathRect: width = (random().nextInt(89) + 1) * DEGREES_TO_RADIANS, then builder.buffer(width); the only upstream caller of the builder's buffer)
The finding in 30 seconds
  • lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dShapeFactory.java:114-117, :175-197, :204-213, :221-224, :278-286, :303-320, :383-407 (main a9779ef3c193)
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoPathFactory.java:30-44; lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoStandardPath.java:65-86, :1456-1458
  • lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dRectangleShape.java:169-173; lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dSpatialContextFactory.java:25-53
Who is affected
  • lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dSpatialContextFactory.java:25-53; lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dShapeFactory.java:204-213, :383-407
  • lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/GeoStandardPath.java:272-275 (write serializes the cutoff angle)
Why it looks unintended
  • lucene/CHANGES.txt:7866-7868; lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/package-info.java:18
  • lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/ShapeRectRelationTestCase.java:159-186
  • lucene/spatial-extras/src/java/org/apache/lucene/spatial/spatial4j/Geo3dShapeFactory.java:82-85 (setCircleAccuracy javadoc), :211 (lineString(List, double) passes through)
Suggested fix
  • lucene/spatial-extras/src/test/org/apache/lucene/spatial/spatial4j/ShapeRectRelationTestCase.java:19, :159-186
  • lucene/CHANGES.txt:268, :331 (section names)
History
  • lucene/CHANGES.txt:7866-7868

Other statements trace to the hunt's working files (models, logs, reviews and briefs), which are not published.

Repro files

About the diagrams

Built 2026-11-06 from the hunt's record of this finding and its doc file. Lucene main under test in this hunt: ea57360912ed. Every listed finding was re-run on Lucene main e1faeba08bde on 2026-11-02.