Geo3D distance sorts drop hits from later pages of searchAfter: compareTop compares meters with planet units

Geo3DDocValuesField.newDistanceSort and its six sibling factories (path, outside distance, outside box, outside polygon, outside large polygon, outside path) document that each hit's FieldDoc value is "a Double instance with the distance in meters", and IndexSearcher.searchAfter tells you to pass the previous page's bottom hit back as the cursor. Inside, the two comparators keep distances in planet units (distances on a planet scaled to mean radius 1): value(slot) multiplies by planetModel.getMeanRadius() to report meters, but compareTop compares the meters cursor with the raw planet-unit distance (Geo3DPointDistanceComparator.java:158-161, Geo3DPointOutsideDistanceComparator.java:128-131). On WGS84 and CLARKE_1866 (mean radius about 6,371 km) a cursor beyond a few meters (about 3.2 m for circle and outside sorts; a long path sort can need more) is larger than every finite planet-unit distance, so the paging collector skips every such document as "Already collected on a previous page". Every later page is then empty or holds only documents whose sort value is +Infinity; a smaller positive cursor gives a partial wrong page. The bug does not change totalHits, and nothing fails. PlanetModel.SPHERE pages correctly. The mismatch came with the feature (LUCENE-7212, 6.1.0) and is in every release through 10.5.2 and in main (executed on nine releases; compareTop and setTopValue unchanged in all 81 release tags scanned). Converting the document distance in compareTop exactly as value() does fixes it.

• Geo3D sorts: Geo3DDocValuesField (module lucene/spatial3d) has seven sort factories: newDistanceSort, newPathSort, newOutsideDistanceSort, newOutsideBoxSort, newOutsidePolygonSort, newOutsideLargePolygonSort and newOutsidePathSort. Each says "The value returned in FieldDoc for the hits contains a Double instance with the distance in meters" (Geo3DDocValuesField.java:167-168, :197-199, :233-234, :264-265, :297-298, :323-324, :347-348). The first two use Geo3DPointDistanceComparator, the five outside sorts Geo3DPointOutsideDistanceComparator.
• Planet units: spatial3d computes on a planet scaled to mean radius 1; a radius in meters becomes radiusMeters / planetModel.getMeanRadius() (Geo3DUtil.java:159). The mean radius is 6,371,008.77 m for WGS84 (PlanetModel.java:35, :142) and 1.0 for PlanetModel.SPHERE (:31).
• searchAfter: "Finds the top n hits for query where all results are after a previous result"; "By passing the bottom result from a previous page as after, this method can be used for efficient 'deep-paging'" (IndexSearcher.java:682-686). The collector gives the cursor's sort values to setTopValue (TopFieldCollector.java:249-254) and skips every document whose compareTop is positive as "Already collected on a previous page" (:275-284).

1. copy(slot, doc) stores values[slot] = computeMinimumDistance(doc), an arc distance in planet units (Geo3DPointDistanceComparator.java:137-139, :163-182).
2. value(slot) returns values[slot] * planetModel.getMeanRadius(), meters, as documented (:152-156).
3. setTopValue stores that meters value unchanged (:91-94).
4. compareTop(doc) returns Double.compare(topValue, computeMinimumDistance(doc)) (:158-161): meters on the left, planet units on the right. Geo3DPointOutsideDistanceComparator has the same four methods (:69-72, :106-109, :122-126, :128-131).
5. A document is skipped whenever its planet-unit distance is below the cursor's number of meters. For circle and outside sorts planet-unit distances are at most about pi, so a cursor beyond about 3.2 m skips every document with a finite distance; a path sort's distance includes the length of the path before the segment it is measured on (GeoStandardPath.java:869-885, :1538-1549), so a long path raises that threshold. Documents whose sort value is +Infinity (no location, which sorts last, or outside a distance or path sort's shape) still compare correctly and survive. A cursor of exactly 0.0 or +Infinity happens to be right; a smaller positive cursor gives a partial wrong page.

Six documents at latitudes 3-8, longitude 0, newDistanceSort("p", 0, 0, 2_000_000, WGS84):

Page 1 is [0, 1]; the cursor is 444,778.37. compareTop compares 444,778.37 with 0.0873, 0.1047, 0.1222 and 0.1396: always positive, so page 2 is [] instead of [2, 3, 4, 5]. The core twin, LatLonPointDistanceComparator, applies the same conversion in value() and in compareTop (LatLonPointDistanceComparator.java:196-204).

• Who is exposed. Users of the seven Geo3D sort factories (module lucene/spatial3d, opt-in) who page with searchAfter from the previous page's FieldDoc, on WGS84 (the default planet of Geo3DPoint) or CLARKE_1866. Before 8.5.0 the factories took no planet argument and always used WGS84.
• No special setup. Default codec and IndexWriterConfig; one segment or several (an independent repro used 2 segments); single-threaded or concurrent search (4 slices in the security probe); any index size.
• Silent. No exception, and the bug does not change totalHits (countHit() runs before compareTop), so an application shows "21 hits" and an empty or short page 2. Across 14 sort and query combinations of one probe, 198 expected hits went missing while every returned hit was correct.
• Deterministic. Every later page, every time, once the cursor is beyond a few meters (for circle and outside sorts; a long path sort can need more).
• Not affected. PlanetModel.SPHERE; plain top-N search and TopDocs.merge, which never call compareTop; LatLonDocValuesField.newDistanceSort.
• How likely. Plausible: no upstream test sorts with these fields, and a code search of the default branches of Elasticsearch, OpenSearch and Solr found no use (code search can miss uses); who uses them is otherwise unknown. An application that does page them loses hits on later pages.
• Workaround until fixed. For a point-distance sort, LatLonDocValuesField.newDistanceSort pages correctly; it has no path or outside-shape sorts and measures haversine distance on a sphere, not on the Geo3D ellipsoid. Otherwise fetch a larger top-N instead of paging. Dividing the cursor by the mean radius works on today's code but breaks once the fix lands. A paging loop should stop on an empty page, not when it has seen totalHits hits: that keeps it from running on, but does not bring the lost hits back.

# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
    => java.lang.AssertionError: hits on page 2 expected:<1> but was:<0>
       org.junit.Assert.fail(Assert.java:89)
       org.junit.Assert.failNotEquals(Assert.java:835)
       org.junit.Assert.assertEquals(Assert.java:647)
       org.apache.lucene.spatial3d.TestGeo3DDistanceSortSearchAfter.testSearchAfterReturnsNextHit(TestGeo3DDistanceSortSearchAfter.java:54)

• The documented unit is meters. All seven factories say so (Geo3DDocValuesField.java:167-168 and the six others); the class is public and has no @lucene.experimental tag (:51).
• searchAfter takes that value back. "By passing the bottom result from a previous page as after, this method can be used for efficient 'deep-paging'" (IndexSearcher.java:685-686).
• The comparator contract. FieldComparator<T>.setTopValue(T) takes the type value(slot) returns and is "only called for searches that use searchAfter (deep paging)" (FieldComparator.java:70-83). LeafFieldComparator.compareTop "should return the same result as FieldComparator#compare(int,int) as if topValue were slot1 and the new document were slot 2" (LeafFieldComparator.java:69-80). An independent test fed a document's own value(slot) back through setTopValue and got compareTop = 1 instead of 0.
• The twin converts by design. LUCENE-7212 is titled "Add Geo3DPoint equivalents of LatLonPointDistanceComparator and LatLonPointSortField". For the twin, LUCENE-7103 (#8158, 1660b5630ae) introduced an internal sort key, and @rmuir wrote: 'We "do the rest" of the haversin calculation only when we need the final value: value() and compareTop()'. TestLatLonPointDistanceSort pages with searchAfter (:250-261). The Geo3D port converts in value() only.
• Precedent. LUCENE-9263, "Fix wrong transformation of distance in meters to radians in Geo3DPoint", is listed under 8.5.0 Bug Fixes (lucene/CHANGES.txt:5980, :6012) and changed value() in both comparators, not compareTop.
• Never tested. No test source references Geo3DDocValuesField or its sort fields, and the 49 comments of #8267 never mention searchAfter, compareTop, paging or units.
• Effect size. Page 2 holds 0 of 4 expected hits; with 13 documents in 2 segments and page size 3, paging returns [0, 1, 2, 12] instead of [0..12].

The strongest case for "not a bug": LUCENE-7212 (#8267) is still open, and its author's last comment lists tests as still needed; there is no CHANGES entry and no test, so the sorts could be called an unfinished, unannounced feature. FieldComparator is @lucene.experimental (FieldComparator.java:56), and so is PlanetModel (PlanetModel.java:26), which every factory takes from 8.5.0 on. Planet units are a deliberate hot-path choice, and a cursor divided by the mean radius by hand pages correctly, so one could claim compareTop is meant to take planet units.

Why it fails: the factories are public, documented in meters and shipped in every release since 6.1.0 (before 8.5.0 they took no PlanetModel and used WGS84); nothing documents a cursor conversion or a searchAfter restriction; the comparator contract requires a cursor taken from value() to compare equal to its own document; the twin converts on purpose; and a unit error in the same comparators was already fixed as a bug. Planet units inside the comparator are fine; only the missing conversion in compareTop is the defect. Verdict: not intended, a defect. LUCENE-7212 is still open and the sorts have no tests.

One fix was built and validated. It changes results for anyone who divided cursors by the mean radius by hand to work around the bug (undocumented, unlikely), which is worth a CHANGES line. It fixes the units only: multi-valued documents are still read twice per competitive hit, a separate bug in the same comparators with its own fix. Until a fix ships, LatLonDocValuesField.newDistanceSort pages correctly.

One line in each of Geo3DPointDistanceComparator.compareTop and Geo3DPointOutsideDistanceComparator.compareTop (+4/-2 with a comment).

   @Override
   public int compareTop(int doc) throws IOException {
-    return Double.compare(topValue, computeMinimumDistance(doc));
+    // topValue comes from value(slot), in meters: convert the document's distance the same way
+    return Double.compare(topValue, computeMinimumDistance(doc) * planetModel.getMeanRadius());
   }

Trade-offs

• Applies exactly the multiplication value(slot) applies, so the cursor's own document compares equal (0) and the collector's tie rule (doc <= afterDoc, TopFieldCollector.java:276) keeps working for documents at the same distance; +Infinity stays +Infinity.
• One multiplication per compareTop call, only on searchAfter pages, next to a doc-values decode and a distance computation. Not measured.
• Changes results for code that divided cursors by the mean radius by hand (undocumented).
• Units only: the separate multi-valued re-read bug in the same comparators needs its own fix; the hunt's random paging harness (seed 3, 1,100 trials, up to 40 documents) finds 0 mismatches in 7,248 checks with both fixes and 2,453 with only the other one; this fix alone leaves 1,070 mismatches and 853 exceptions.

What the checks validated

• Scratch worktree of main a9779ef3c193, gradle, JDK 25: stock + the new test, 8 of 9 fail with assertions on and off; + fix, the new test passes 9 of 9 with assertions on and off.
• + fix, the whole lucene/spatial3d test task: 20 suites, 190 tests (181 existing + 9 new), 3 skipped, 0 failures, with assertions on and off; :lucene:spatial3d:check -x test (ecjLint, forbiddenApis, checkGoogleJavaFormat) passes.
• Standalone runner with the fix shadowed: 9 of 9 with -ea and -da.
• Earlier: the original repro 6 of 6, the independent reproducer's tests 10 of 10, the security probe drops 0 hits.
• No existing upstream test sorts with these fields, so the new test is the coverage.

topValue = value.doubleValue() / planetModel.getMeanRadius(); saves the per-document multiplication. Not built.

Trade-offs

Division does not exactly undo multiplication: with the WGS84 mean radius, (x * R) / R != x for 144,779 of 1,000,000 random distances in [0, 3.2]. For those the page-1 bottom hit no longer compares equal to its own cursor, so it could come back on page 2, or a document at the same distance with a higher doc id could be skipped (reasoned, not run in Lucene).

Multiply in copy() and compare meters everywhere. Not built.

Trade-offs

setBottom and compareBottom work in planet units (bounding boxes from getDistanceBounds(..., bottomDistance), Geo3DPointDistanceComparator.java:77-134), so they would all need the conversion: a bigger change for the same result.

Rejected.

Trade-offs

Contradicts the factories' meters javadoc and the comparator contract, and every caller would have to know about it.

Method: testing (metamorphic relations), then code reading.

• First signal. While probing a different Geo3D bug (LFV-274, outside distances of small circles), a control that paged a plain WGS84 newDistanceSort five hits at a time returned 5 of 36 documents and then an empty page. It was recorded as an untriaged side finding.
• Triage. A later hunt unit on sort-field pagination built a random harness with metamorphic relations, among them R1 (searchAfter pages concatenate to the full sorted result) and R4 (the same with concurrent search). The Geo3D sorts broke R1 and R4 on WGS84 only. Reading the comparator showed value() converting to meters and compareTop not converting; with both Geo3D comparator fixes (this one and the separate multi-valued re-read fix) the harness found 0 mismatches in 7,248 checks (seed 3, 1,100 trials).
• An AI finder then reduced it to a 6-test public-API JUnit, and independent AI tools and reviewers from another model family reproduced, reviewed and judged it (see the checks).