LatLonShape WITHIN a circle that crosses the dateline matches a line that runs through longitude 0

LatLonShape.newDistanceQuery(field, WITHIN, circle) matches a document only if every indexed segment and triangle of its shape is inside the circle, and asks Circle2D.containsLine / containsTriangle for each. When the circle's bounding box crosses the dateline, Circle2D tests points against two longitude pieces, [minLon, 180] and [-180, maxLon], and these two methods return true once every vertex passes that test, without checking that the vertices lie in the same piece. Lucene's indexed edges are straight lines in lon/lat that do not wrap at ±180, so an edge with a vertex in each piece runs across the whole map, through longitude 0, far outside the circle, and the document is still returned as WITHIN. In the one-test repro a line from (0, -179.5) to (0, 179.5), which Lucene itself places through (0, 0), is WITHIN a 100 km circle around (0, 180): 1 hit instead of 0, silently, with assertions on and off. Lucene's own dateline-crossing box query just around the circle, and two polygons split at the 180th meridian over the same box, answer 0. The trigger is contrived (such a shape has an edge that spans at least 360 degrees minus the circle's longitude width, about 358 for the repro's circle, and whoever indexes it usually meant the short path), so the priority is Low. Present since the API was added (8.5.0); not fixed upstream. Requiring all vertices to be in the same piece fixes the repro (the unsplit shape now gives 0) and keeps the geo and shape suites green; the split form of the shape still matches.

1. WITHIN uses per-edge containment. SpatialQuery uses within() as the leaf predicate for WITHIN (document/SpatialQuery.java:123-135). For an indexed LINE it calls component2D.containsLine (document/LatLonShapeQuery.java:139-146, call at :145), for a TRIANGLE containsTriangle (:147-156, call at :155).
2. Contract. Component2D: "return true if this component2D contains the provided line" and "... triangle" (geo/Component2D.java:73, :84).
3. Two pieces. HaversinDistance sets crossesDateline = rectangle.minLon > rectangle.maxLon (geo/Circle2D.java:418); contains(x, y) then accepts a point only inside [minLon, 180] or [-180, maxLon] and within the haversine radius (:426-447, the pieces at :429-437). For Circle(0, 180, 100 km): [179.1007, 180] and [-180, -179.1007], latitudes ±0.8993.
4. The defect (fix site). containsLine (geo/Circle2D.java:110-123) runs the bounding-box disjoint test (:119-121), then return contains(aX, aY) && contains(bX, bY); (:122). containsTriangle (:126-141) does the same for three vertices (:140). Nothing checks that the vertices are in the same piece.
5. Of the edge tests, only intersectsLine has a dateline step. HaversinDistance branches on crossesDateline in its point and box tests (contains, disjoint, within, getMinX, getMaxX: geo/Circle2D.java:428, :463, :497, :531, :540), and intersectsLine retries with the centre shifted by 360 degrees for a crossing circle (:449-459, the retry at :454-457; LUCENE-9250, #10290). containsLine and containsTriangle got no such step.
6. The box query does it right. Its containsLine / containsTriangle require all vertices on the same side when the box wraps (document/SpatialQuery.java:911-921, :923-933), and Rectangle2D.create builds a crossing box from two components (geo/Rectangle2D.java:299-316).
7. Why the tests are silent. The random shape tests use the query component's own containsLine / containsTriangle as the WITHIN oracle (BaseSpatialTestCase.java:769, :781).

• Line A = (0, -179.5), B = (0, 179.5): one segment covering 359 degrees of longitude. A and B are each 55.6 km from (0, 180); the short path between them is 111.2 km.
• Lucene's edge passes (0, 0), 20,015 km from the centre. Only the 0.4 degrees next to the dateline at each end, 0.8 of 359 degrees (0.22% of the edge), are inside the circle.
• Component level: containsLine(-179.5, 0, 179.5, 0) is true in both orders while contains(0, 0) is false; a triangle with vertices (-179.5, 0), (179.5, 0), (179.5, 0.1) gives containsTriangle true.
• Query level: WITHIN Circle(0, 180, 100 km) 1; INTERSECTS Circle(0, 0, 100 km) 1; WITHIN the dateline-crossing box lat [-0.9, 0.9], lon [179.1, -179.1] (rounded out from the circle's box) 0; WITHIN the two polygons split at the 180th meridian 0.
• The documented split form of the line (two parts meeting at the 180th meridian): WITHIN the circle 1, before and after the fix.

• Who is hit. Users of LatLonShape.newDistanceQuery (or newGeometryQuery with a Circle) with WITHIN on line or polygon fields, when the circle's bounding box crosses the dateline (the circle reaches past ±180 but not a pole, since a circle that reaches a pole gets the full longitude range and no crossing, geo/Rectangle.java:156-171; 100 km around (0, 180) in the repro) and the indexed shape has vertices inside the circle on both sides of the dateline.
• Default, not opt-in. Default codec, IndexWriterConfig and searcher; one document; assertions on or off; no error and nothing wrong in the index.
• Effect. A false WITHIN match: in the repro 99.8% of the edge's longitude span is outside the circle. The same document also INTERSECTS a 100 km circle at (0, 0), so Lucene's answers contradict each other.
• How likely: contrived. Such a shape has an edge that spans at least 360 degrees minus the circle's longitude width (about 358 for the repro's 100 km circle), and every vertex lies inside the circle, so close to the dateline (within 0.9 degrees of it in the repro). Whoever indexes it by mistake usually meant the short path across the dateline, and for that shape the hit is the answer they want. No public report was found in the recorded searches (2026-11-02).
• Not affected. Point documents, shapes whose vertices are all on one side, circles that do not cross the dateline, XYShape circles (Cartesian, no dateline), and the doc-values shape query, whose only public entry (LatLonShape.newSlowDocValuesBoxQuery) takes a box, not a circle.

The hunt's minimal repro with one change: the comment's trailing "; actual 1" is dropped, because that line was 101 characters and failed google-java-format. With the change the test passes :lucene:core:check -x test. The test framework still randomizes codec, locale and time zone per seed; every recorded run, over several seeds, failed the same way.

Executed: gradle on main 5adff78bb168 with assertions on and off: fails at the WITHIN assertion (line 43); with the fix it passes. The hunt's standalone runs, a separate re-run and the audit's re-run of the earlier form (an INTERSECTS check first, then the same WITHIN assertion) fail the same way.

Use the same test as the regression test.

cp $REPRO/TestLatLonShapeWithinDatelineCircle.java $LUCENE/lucene/core/src/test/org/apache/lucene/document/
cd $LUCENE && ./gradlew :lucene:core:test --tests TestLatLonShapeWithinDatelineCircle
# fails the same way with -Ptests.asserts=false

TestLatLonShapeWithinDatelineCircle > testLineThroughLon0IsNotWithinDatelineCircle FAILED
    java.lang.AssertionError: expected:<0> but was:<1>
        at __randomizedtesting.SeedInfo.seed([5EED1134:11C70AACBE40AD10]:0)
        at org.apache.lucene.document.TestLatLonShapeWithinDatelineCircle.testLineThroughLon0IsNotWithinDatelineCircle(TestLatLonShapeWithinDatelineCircle.java:43)

# with the fix: BUILD SUCCESSFUL, the test passes with assertions on and off

• Internal contradiction. One document is WITHIN Circle(0, 180, 100 km) and INTERSECTS Circle(0, 0, 100 km); their centres are 20,015 km apart and each radius is 100 km.
• The edge model is Lucene's own. Indexed edges do not wrap at ±180: as Ignacio Vera wrote on #9767 (LUCENE-8721, 2019), "a line on the dateline will never intersects any other shape on the other side but I see that more like a feature". INTERSECTS follows that model; WITHIN does not.
• Contracts. "return true if this component2D contains the provided line" (geo/Component2D.java:73, triangle :84). The Circle javadoc: "Dateline crossing is handled automatically" (geo/Circle.java:34, since 10.3.0, #14534); only "very large circles or circles near poles" are flagged (:33), and a 100 km circle on the equator is neither.
• Same commit, two behaviours. LUCENE-9273 (f914e08b364) wrote today's vertex-only Circle2D containment and, in the same change, wrote the box query's containsLine / containsTriangle, which keep the box's existing rule that all vertices lie on the same side of a box that wraps the dateline (before the change the box query already tested a triangle against each side separately; now document/SpatialQuery.java:911-933).
• The documented form is unaffected. The split form of the shape (two parts that meet at the 180th meridian) returns 1 for WITHIN the circle before and after the fix. The fix does change unsplit shapes like the repro's from 1 to 0: a user who stored one meaning the short path loses that hit. That is the fix's compatibility cost.

Strongest case for "intended": whoever indexes (0, -179.5)-(0, 179.5) almost always means the 1-degree path across the dateline, which does lie in the circle; the Polygon javadoc says polygons "cannot cross the 180th meridian" (geo/Polygon.java:36); newLineQuery / newPolygonQuery say "does not support dateline crossing" (document/LatLonShape.java:286-301); Circle is @lucene.experimental (geo/Circle.java:47).

Why it fails: the indexed Line has no dateline precondition, and in Lucene's model it does not cross the dateline at all: Lucene places its edge through lon 0 and answers INTERSECTS accordingly. The dateline notes concern polygons and query-side geometries. @lucene.experimental allows API changes, not wrong answers. Lucene's own wrapping box rejects the same document. Every verifier, two of them from another model family (OpenAI GPT-5.6, which rated the argument weak), and the adjudication (at medium confidence) rejected it; a later fact check by the same model family rated it moderate as a "won't fix" argument (rare trigger, compatibility cost) and weak as a reason the current answer is right. Verdict: not intended, a defect. The split form of the shape is unaffected by the fix, which only turns the unsplit shape's 1 into 0.

If two contained vertices lie in different pieces, the lon/lat-straight edge between them crosses the whole gap, which is outside the circle's bounding box, so by this argument rejecting it adds no false negative (an argument over Lucene's edge model, not an exhaustive test; the suites found none). One fix was built and tested in a scratch worktree of 5adff78bb168; the alternatives are listed with their trade-offs.

Add a samePiece(aX, bX) default method to Circle2D.DistanceCalculator returning true (Cartesian calculator and non-crossing circles unchanged), override it in HaversinDistance, and require it in containsLine and, for a-b and b-c (which covers a-c), in containsTriangle. Full patch: +19/-2 in geo/Circle2D.java.

// containsLine (Circle2D.java:122)
return contains(aX, aY) && contains(bX, bY) && calculator.samePiece(aX, bX);

// HaversinDistance
@Override
public boolean samePiece(double aX, double bX) {
  return crossesDateline == false || (aX >= rectangle.minLon) == (bX >= rectangle.minLon);
}

Trade-offs

Exact for the case it changes; shapes with all vertices in one piece and the split form are answered as before. Behaviour change: unsplit shapes with contained vertices on both sides of the dateline go from WITHIN 1 to 0 (executed for the repro's line and a polygon with the same vertex longitudes), so a user who stored one meaning the short path loses that hit. Cost: one or two comparisons per contained segment or triangle. Not run: the full Lucene test suite. Variant (not run): reject when the segment's bounding box, which containsLine already receives, spans the gap (minX <= maxLon && maxX >= minLon); same result, still needs a calculator method.

What the checks validated

Scratch worktree of 5adff78bb168, gradle, OpenJDK 25.0.3: the repro fails stock and passes fixed, with assertions on and off; 53 geo and shape suites (org.apache.lucene.geo.*, document.TestLatLon*, TestXY*, TestShape*, search.TestLatLon*, TestXY*; 585 tests, 40 skipped, 0 failures, random root seed; not the whole Lucene suite) pass; :lucene:core:check -x test (google-java-format, ecj lint, forbidden APIs) passes. The hunt's 7-case component matrix, the intended-lens review's line and polygon tests and the devil's-advocate probe (unsplit line WITHIN 0, split form 1) also pass with it.

As Rectangle2D.create does for a crossing box, represent the circle as the circle clipped to each longitude piece and combine the two.

Trade-offs

Closer to how boxes work, but a larger change that touches every Circle2D relation. Not implemented or run.

State that indexed shapes queried WITHIN a dateline-crossing circle must be split at the dateline.

Trade-offs

Not recommended: leaves the contradiction with INTERSECTS and with the box query, and with "Dateline crossing is handled automatically".

Method: non-formal (code reading of Circle2D, witnessed by a one-test public-API JUnit on the stock main jars).

• The reading compared the dateline handling of the circle's methods: contains splits a crossing circle into two longitude pieces, intersectsLine retries with the centre shifted by 360 degrees, but containsLine / containsTriangle only test each vertex. A JUnit with a line across the gap witnessed the false WITHIN match on main, 10.6.0 and 9.12.3, and a same-piece fix sketch made it pass.
• Then an independent reproducer wrote its own test with other numbers, cross-model reviews (OpenAI GPT-5.6, refute and intended lenses, executed) wrote and ran their own tests, a devil's-advocate verifier compared the unsplit and split shapes against the circle, the box and the split polygons, and an adjudication re-ran a test on clean main. An audit and a final repro check followed.
• Earlier hunt coverage of the geo shapes had listed haversine circles as not checked.