Code that relates a shape doc value to a query box and then asks it for its bounding box gets the box of a piece of the shape instead.
relate() the getters report the whole line's box (lat 0..20, lon 0..20); after a relate() near (20, 20) they report the box of the segment (10,10) -> (20,20) it walked into (lat 10..20, lon 10..20), while getBoundingBox() still reports the whole box. Select the still to open it at full size. Generated with AI, checked against the code.- Build a
LatLonShapeDocValuesfor the line (lat, lon) (0,0) -> (10,10) -> (20,20) withLatLonShape.createLatLonShapeDocValues(...). Its getters give the shape's box, lat 0..20 and lon 0..20:getEncodedMinX()= 0,getEncodedMinY()= 0. - Call
relate()with the small box lat [19, 21], lon [19, 21] at the line's far end. It answersCELL_CROSSES_QUERY, which is correct. You expect the four getters, documented as "the shape's bounding box", to be unchanged. - Instead
getEncodedMinX()is 119304647 (longitude 10) andgetEncodedMinY()is 238609294 (latitude 10): the box of the segment (10,10) -> (20,20), the child noderelate()walked into. No error, assertions on or off.getBoundingBox()on the same object still reports lat 0..20, lon 0..20. - Why: the object keeps no copy of its box. The getters read the reader's one reusable scratch box, and
relate()overwrites that box at each child node it walks into (a child skipped as disjoint leaves it alone). A laterrelate()answered at the root (say a far-away box) puts the right values back, so the answer depends on call history.
Only code that uses the experimental LatLonShapeDocValues / XYShapeDocValues object directly, calls relate() with a query that crosses the shape's box, and then reads getEncodedMinX/MinY/MaxX/MaxY on the same object. Lucene's own queries, relate() answers and getBoundingBox() are unaffected; no setting or scale is involved. Rare.
Mechanism, terms and numbers493 words
LatLonShapeDocValues and XYShapeDocValues (@lucene.experimental) wrap one shape's doc-value bytes; client code gets one from the public factories LatLonShape.createLatLonShapeDocValues(BytesRef) / XYShape.createXYShapeDocValues(BytesRef) (since 9.5.0) and can call the public relate(Component2D) and the four public getters getEncodedMinX/MinY/MaxX/MaxY, documented as the shape's bounding box. The getters do not keep their own copy: they read the reader's single reusable scratch box, which relate() also uses as working space while it walks down the shape's tree. So after a relate() that walks into child nodes, the getters return the box of the last one it did not skip as disjoint. getBoundingBox(), computed once in the constructor, stays right, so one object reports two different boxes. Lucene's own queries are unaffected (they build a fresh object per document and never read the getters). Present since the feature was added (d7fd48c9502, 2022); the fix is to copy four ints at construction, like the centroid.
- One scratch box.
ShapeDocValues.java:578: theReaderkeepsprivate final BBox bbox; // scratch bbox instance.readBBox()(:599-606) andresetBBox(..)(:609-612) both returnbbox.reset(..), which overwrites the four fields and returnsthis(:663-670). - Kept by reference. The
ShapeComparatorconstructor storesthis.boundingBox = dvReader.readBBox();(:691) in aprivate final SpatialQuery.EncodedRectanglefield (:680): the scratch object itself, not a copy. The public getters (:105-123) read it throughgetMinX..getMaxY(:702-716). - Overwritten by the walk.
relate(Component2D)(:743-802) re-reads the header box into the scratch box (:750). When the shape's box crosses the query and the root component does not decide it, it recurses, and every visited child that is not skipped as disjoint (:826-831) callsdvReader.resetBBox(tMinX, tMaxX, tMinY, tMaxY)(:836). Nothing restores the header box. - The rest is copied.
version,numberOfTerms,centroidX/YandhighestDimensionare copied into final fields (fields:678-683, assigned at:688-694); only the box (:680,:691) is held by reference. - The other box stays right.
getBoundingBox()iscomputeBoundingBox(), run once in the constructor (:80,:92) from the four getters (LatLonShapeDocValues.java:66-71,XYShapeDocValues.java:67-72), before anyrelate().
Line (0,0) -> (10,10) -> (20,20), numberOfTerms() = 2.
| call | relate() answer | minX | maxX | minY | maxY |
|---|---|---|---|---|---|
| fresh object | 0 | 238609294 | 0 | 477218588 | |
relate lat [19, 21], lon [19, 21] | CELL_CROSSES_QUERY | 119304647 | 238609294 | 238609294 | 477218588 |
relate lat [40, 41], lon [40, 41] (answered at the root) | CELL_OUTSIDE_QUERY | 0 | 238609294 | 0 | 477218588 |
relate lat [4, 6], lon [4, 6] (answered by the root segment) | CELL_CROSSES_QUERY | 0 | 238609294 | 0 | 477218588 |
119304647 = encodeLongitude(10), 238609294 = encodeLongitude(20) = encodeLatitude(10), 477218588 = encodeLatitude(20). getBoundingBox() prints Rectangle(lat=0.0 TO 19.99999998137355 lon=0.0 TO 19.99999998137355) in each of the first three rows (the probe does not print it in the fourth; it is computed once, in the constructor).
Exposure and likelihood172 words
- Who is exposed. Client code that gets the object from
LatLonShape.createLatLonShapeDocValues(BytesRef)orXYShape.createXYShapeDocValues(BytesRef)(LatLonShape.java:350-352,XYShape.java:280-282, public since 9.5.0), calls the publicrelate()(ShapeDocValues.java:163-165) and afterwards reads the encoded getters, for example a custom collector that relates per-document shapes and then reports their bounds. - Trigger. The query must cross the shape's box, the root component must not decide it, and some child node must be visited (a shape with two or more tessellation components). A
relate()answered at the root or by the root component leaves the getters right (executed). - Not exposed. Lucene's shape doc-values queries build a fresh object per document (
BaseShapeDocValuesQuery.java:78,LatLonShapeDocValuesQuery.java:42-43) and never read the getters; the only in-tree callers of the getters are the twocomputeBoundingBox()methods, which run before anyrelate(). - Silent. No exception and no assertion, with
-eaand-da; the value is plausible (a real box inside the shape's box). - How likely. Rare: no public report was found since the factories became public in 9.5.0 (2023).
One JUnit test for lucene/core, public API only, no index, single thread, fixed input: build the three-point line, read getEncodedMinX(), call relate() with a box around (20, 20), read it again. It fails the same way with assertions on and off.
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.lucene.document;
import org.apache.lucene.geo.LatLonGeometry;
import org.apache.lucene.geo.Line;
import org.apache.lucene.geo.Rectangle;
import org.apache.lucene.tests.util.LuceneTestCase;
public class TestShapeDocValuesBBoxAfterRelate extends LuceneTestCase {
public void testBoundingBoxUnchangedByRelate() throws Exception {
Line line = new Line(new double[] {0, 10, 20}, new double[] {0, 10, 20});
LatLonShapeDocValues dv =
LatLonShape.createLatLonShapeDocValues(
LatLonShape.createDocValueField("f", line).binaryValue());
int minX = dv.getEncodedMinX();
dv.relate(LatLonGeometry.create(new Rectangle(19, 21, 19, 21)));
// expected: the shape's bounding box minX is unchanged by relate(); actual: a tree node's minX
assertEquals(minX, dv.getEncodedMinX());
}
}
cp $REPRO/TestShapeDocValuesBBoxAfterRelate.java $LUCENE/lucene/core/src/test/org/apache/lucene/document/
cd $LUCENE && ./gradlew :lucene:core:test --tests TestShapeDocValuesBBoxAfterRelate
# fails the same with -Ptests.asserts=false; the input is fixed (no randomness); it failed under every seed it was run with
- Expected
getEncodedMinX()is still 0 afterrelate(), as before it.- Observed
AssertionError: expected:<0> but was:<119304647>: longitude 10, a tree node's minX.
More files, environment, full output185 words
This is the hunt's minimal repro, unchanged. Use the same test as the regression test. The same calls also work from a client package (several of the independent checks ran such tests), and XYShapeDocValues fails the same way (the independent reproducer's XY method), so one test is enough.
Executed: gradle on main 5adff78bb168 with assertions on (seed 5EED1130) and off (5EED1131): fail; the 10.6.0 and 9.5.0 release jars, assertions off: fail, same message; the hunt's standalone runs on main with -ea and -da: fail. With the fix: pass.
| lucene | main 5adff78bb168 (12.0.0-SNAPSHOT); the same test also fails on the 10.6.0 and 9.5.0 release jars |
|---|---|
| jdk | OpenJDK 25.0.3 (build 25.0.3+9-LTS) |
| os | Linux, aarch64 (plain Java; only this platform was run) |
cp $REPRO/TestShapeDocValuesBBoxAfterRelate.java $LUCENE/lucene/core/src/test/org/apache/lucene/document/
cd $LUCENE && ./gradlew :lucene:core:test --tests TestShapeDocValuesBBoxAfterRelate
# fails the same with -Ptests.asserts=false; the input is fixed (no randomness); it failed under every seed it was run with
TestShapeDocValuesBBoxAfterRelate > testBoundingBoxUnchangedByRelate FAILED
java.lang.AssertionError: expected:<0> but was:<119304647>
at __randomizedtesting.SeedInfo.seed([5EED1130:B89269ED6AB2EFEA]:0)
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:633)
at org.apache.lucene.document.TestShapeDocValuesBBoxAfterRelate.testBoundingBoxUnchangedByRelate(TestShapeDocValuesBBoxAfterRelate.java:34)
# the same with -Ptests.asserts=false- Each getter's javadoc says it returns a value "for the shape's bounding box"; nothing says
relate()changes it. - The same object's
getBoundingBox()is built from these getters at construction and stays right: one object, two boxes. - Every other header value (version, term count, centroid, dimension) is copied into a final field; only the box references the scratch object.
The full argument116 words
- Contract.
ShapeDocValues.java:105,:110,:115,:120: "returns the min x [min y / max x / max y] value for the shape's bounding box".relate()has no javadoc; nothing documents it as changing the getters. - Tests. No test in the repository reads the four getters;
TestShapeDocValueschecksgetBoundingBox()only (grep oflucene/at5adff78bb168). - History. The alias came with the feature (
d7fd48c9502, LUCENE-10654, #1017). Issue #11752, "Enable Relate Interface of Shape Doc Values to be used by clients of Lucene", and its PR #11753 added the public factories so that client code can callrelate(). None of #1017, #11690, #11752 or #11753 discusses the getters or the scratch box.
The devil's-advocate check131 words
The strongest "intended" case (rated moderate by the cross-model reviews and the devil's-advocate verifier): the scratch reuse is deliberate, so the traversal allocates nothing per visited node; the object is a per-document, query-time view; the classes are @lucene.experimental, the superclass is package-private and Component2D is @lucene.internal; and getBoundingBox() already gives a stable box.
Why it fails: the fix keeps the scratch reuse in the traversal and copies four ints once per object; @lucene.experimental allows API changes, not a getter that contradicts its own javadoc depending on an earlier call; and #11752 / #11753 made relate() reachable from client code by adding the factories (the public LatLonGeometry.create / XYGeometry.create supply the Component2D). A maintainer may still prefer to document or narrow the getters; the fix section lists that option.
Recommended Keep the header box as four final ints (fix A)
Copy the header box into four final int fields in the ShapeComparator constructor, like centroidX/Y, and let the getters read them; the traversal keeps its scratch box. Validated with gradle on 5adff78bb168: the repro fails on stock and passes with the fix; 10 shape suites (68 tests) and :lucene:core:check -x test pass.
All options and trade-offs3 options ยท 588 words
One fix was built and validated with gradle in a scratch worktree of 5adff78bb168; a second, equivalent one was validated with the standalone runner.
Replace the EncodedRectangle boundingBox field with minX, maxX, minY, maxY, filled from readBBox() in the constructor; getMinX..getMaxY return them.
diff --git a/lucene/core/src/java/org/apache/lucene/document/ShapeDocValues.java b/lucene/core/src/java/org/apache/lucene/document/ShapeDocValues.java
--- a/lucene/core/src/java/org/apache/lucene/document/ShapeDocValues.java
+++ b/lucene/core/src/java/org/apache/lucene/document/ShapeDocValues.java
@@ -677,7 +677,10 @@ abstract class ShapeDocValues {
private final Encoder encoder;
private final byte version;
private final int numberOfTerms;
- private final SpatialQuery.EncodedRectangle boundingBox;
+ private final int minX;
+ private final int maxX;
+ private final int minY;
+ private final int maxY;
private final int centroidX;
private final int centroidY;
private final TYPE highestDimension;
@@ -688,7 +691,12 @@ abstract class ShapeDocValues {
this.version = dvReader.readByte();
assert this.version == VERSION;
this.numberOfTerms = Math.toIntExact(dvReader.readVInt());
- this.boundingBox = dvReader.readBBox();
+ // copy the header box: readBBox() returns the reader's scratch box, which relate() reuses
+ SpatialQuery.EncodedRectangle bbox = dvReader.readBBox();
+ this.minX = bbox.minX;
+ this.maxX = bbox.maxX;
+ this.minY = bbox.minY;
+ this.maxY = bbox.maxY;
this.centroidX = Math.toIntExact(dvReader.readVLong() + Integer.MIN_VALUE);
this.centroidY = Math.toIntExact(dvReader.readVLong() + Integer.MIN_VALUE);
this.highestDimension = TYPE.values()[dvReader.readVInt()];
@@ -700,19 +708,19 @@ abstract class ShapeDocValues {
}
private int getMinX() {
- return this.boundingBox.minX;
+ return this.minX;
}
private int getMinY() {
- return this.boundingBox.minY;
+ return this.minY;
}
private int getMaxX() {
- return this.boundingBox.maxX;
+ return this.maxX;
}
private int getMaxY() {
- return this.boundingBox.maxY;
+ return this.maxY;
}
public TYPE getHighestDimension() {Trade-offs
Nothing extra is allocated (four int fields replace one reference, so the object may be a few bytes larger); no change to the traversal, the format or the public API. As before, one object is not safe for concurrent relate() calls (the reader position moves).
What the checks validated
- stock: the repro fails 1 of 1 with assertions on (seed
5EED1130) and off (5EED1131):expected:<0> but was:<119304647>; - fix A: the repro passes with assertions on and off;
- fix A:
TestShapeDocValues*,TestLatLon*ShapeDVQueries,TestXY*ShapeDVQueries,TestLatLonShapeandTestXYShape(seed5EED1132): 10 suites, 68 tests, 0 failures, 7 skipped; - fix A with the test in the tree:
:lucene:core:check -x test(checkGoogleJavaFormat,ecjLint,forbiddenApis) passes.
At :691, copy the header box into a new SpatialQuery.EncodedRectangle. EncodedRectangle is a public static class in the same package as ShapeDocValues (SpatialQuery.java:803-817), so its protected constructor can be called directly.
diff --git a/lucene/core/src/java/org/apache/lucene/document/ShapeDocValues.java b/lucene/core/src/java/org/apache/lucene/document/ShapeDocValues.java
--- a/lucene/core/src/java/org/apache/lucene/document/ShapeDocValues.java
+++ b/lucene/core/src/java/org/apache/lucene/document/ShapeDocValues.java
@@ -688,7 +688,10 @@ abstract class ShapeDocValues {
this.version = dvReader.readByte();
assert this.version == VERSION;
this.numberOfTerms = Math.toIntExact(dvReader.readVInt());
- this.boundingBox = dvReader.readBBox();
+ SpatialQuery.EncodedRectangle header = dvReader.readBBox();
+ this.boundingBox =
+ new SpatialQuery.EncodedRectangle(
+ header.minX, header.maxX, header.minY, header.maxY, false);
this.centroidX = Math.toIntExact(dvReader.readVLong() + Integer.MIN_VALUE);
this.centroidY = Math.toIntExact(dvReader.readVLong() + Integer.MIN_VALUE);
this.highestDimension = TYPE.values()[dvReader.readVInt()];Trade-offs
Same effect as A; one more small object per instance.
What the checks validated
Shadowed on main: the repro passes with assertions on and off, and the 7 upstream TestShapeDocValues tests pass. The hunt had validated the same copy written as an anonymous subclass, which is not needed.
Javadoc "valid only until the next relate()", or make the getters non-public and point users to getBoundingBox().
Trade-offs
Keeps a getter whose value depends on call history, or changes the API of a public (experimental) class. Not recommended, though a maintainer may prefer it.
The one downstream wrapper the search found, OpenSearch's GeoShapeDocValue, calls relate() and getBoundingBox() but not the encoded getters, so it does not see the wrong values (by reading its source).
A proof-obligation audit of ShapeDocValues' contract checked the getters' promise, "the shape's bounding box", against every state relate() can leave the object in, and found the shared scratch box; a one-test public-API repro witnessed it. An independent reproducer, cross-model reviews, a devil's-advocate verifier and an adjudication confirmed it.
The method in full165 words
Method: semi-formal (a proof-obligation audit of a class's contract, witnessed by an executable repro).
- The obligation: after any sequence of public calls,
getEncodedMinX/MinY/MaxX/MaxYequal the box encoded in the doc value's header. The proof step "the comparator's box is never written after construction" fails, because the field aliases the reader's scratch box andrelate()writes it per visited node; a public-API JUnit witnessed it on main, 10.6.0 and 9.5.0, and copying the header box at construction made it pass. - Then an independent reproducer wrote its own client-package tests (LatLon and XY), cross-model reviews (OpenAI GPT-5.6, refute and intended lenses, executed) wrote and ran their own tests, a devil's-advocate verifier argued the intended-behaviour case with its own XY polygon test, and an adjudication re-ran the tests. An audit and a final repro check followed and reduced the repro to the single test shown here.
- An earlier hunt pass over the shape doc-values tree had checked the getters only on fresh objects and did not see it.
No public report found (searched 2026-11-02): apache/lucene issues, PRs and commits including the migrated JIRA archive, the Lucene mailing lists, the Elasticsearch, OpenSearch and Solr trackers and the web. Manual GitHub searches for the getter names and readBBox / resetBBox were recorded as 0 hits.
Searches3 searches
None of the hits is the same bug. Limits: keyword search only; GitHub search does not index review comments (the comments of the nearest threads were read directly); private reports are not visible. A re-check the same day found no new issue, PR or commit, and ShapeDocValues.java unchanged on upstream main since de3b294be44 (2023-10-10).
| Where | When | How | Found |
|---|---|---|---|
| GitHub issues, PRs and commits (apache/lucene incl. the migrated JIRA archive; downstream repositories) | 2026-11-02 | a scripted prior-report search of 53 queries plus earlier runs of 45 and 27 queries; manual searches for the class names, getEncodedMinX/MaxX/MinY/MaxY, readBBox, resetBBox, ShapeComparator, symptom phrases; open PRs | no report; the getter names and readBBox/resetBBox were recorded as 0 hits each (raw results not saved); none of the open PRs checked touches the shape files |
| Mailing lists and JIRA | 2026-11-02 | 20 queries over dev@, java-user@, issues@ and commits@; 4 JIRA-archive queries | no report |
| Nearest items (read in full) | 2026-11-02 | #1017, #11690, #11752, #11753, #11974, #1066, #13531, #13743 (comments; review comments of #1017) | background only: the feature (#1017 / #11690), making relate() reachable from client code through public factories (#11752 / #11753), other defects in the same classes (#11973 / #11974, #1066) |
Introduced with shape doc values by d7fd48c9502 (LUCENE-10654, #1017, 2022-08-09, shipped in 9.4.0); reachable from client code since the public factories of #11753 (9.5.0). Fails on every executed release from 9.5.0 to 10.6.0 and on main; not a regression; not fixed upstream.
Versions tested6 rows
git blame gives all four defect lines (the scratch box, reset returning this, the alias at :691, the per-node resetBBox at :836) to d7fd48c9502 (Nick Knize; branch_9x copy ddf0d0acf4e, #1064). e7253f112de (#11753, issue #11752, Navneet Verma, 2022-10-28; branch_9x copy a4ef70f0656, #11890) added the public factories, first released in 9.5.0. In 9.4.x no public API returns an instance. The two later commits on the file (fad3108b27b, #11974; de3b294be44, formatting) do not touch these lines; git log -G'readBBox|resetBBox|boundingBox' on the file (main) finds only d7fd48c9502.
| Version or commit | Verdict | Result |
|---|---|---|
main 5adff78bb168 | affected | the test fails 1 of 1 through gradle with assertions on and off |
| 10.6.0, 9.5.0 | affected | this doc's test fails 1 of 1 on the release jars, assertions off |
| 10.5.2, 10.0.0, 9.12.3 | affected | the hunt's own tests fail on the release jars, as on main |
| 9.4.0, 9.4.2 | no client access | the code is there and a package-access variant fails, but no public API returns an instance, so the repro does not compile |
| 9.3.0 | before | no shape doc values |
| main + fix A | fix | the test passes; so do 10 shape and shape doc-values suites |
An independent reproducer, cross-model reviews (OpenAI GPT-5.6, executed) and a devil's-advocate verifier each wrote and ran their own failing test; an adjudication, an audit and a final repro check re-ran them, and all agree on the root cause. This doc's gradle runs reproduced it on main and validated fix A.
Checks and verdicts256 words
Every check that ran saw the getters change after a descending relate() and trace it to ShapeDocValues.java:691 and :836; every one that ran a fix saw it pass.
| Check | Verdict | What it found |
|---|---|---|
| finder | reproduced | One public-API test fails on main (on, off), 10.6.0 and 9.5.0; passes with fix B; TestShapeDocValues 7 of 7 with it. |
| independent reproducer | real | Own client-package polyline tests, LatLon and XY (relate() returns OUTSIDE after walking the tree): fail on main, 10.6.0, 10.5.2, 9.12.3 and 9.5.0; pass with the fix. |
| cross-model review, refute lens | real (high) | OpenAI GPT-5.6, executed: its own XY test fails with assertions on and off, its root-decided control passes; an outside-package test shows the API is reachable from client code. |
| cross-model review, intended lens | real (high) | OpenAI GPT-5.6, executed: its own outside-package test fails and passes with the fix; intended argument moderate, rejected. |
| devil's-advocate verifier | real (medium-high) | Own L-shaped XY polygon test (query box in the empty part of the L) fails on main and on the 10.6.0 jars. |
| adjudication | confirmed (high), Low | Re-ran the reproducer's tests on clean main with assertions on and off: 2 of 2 fail. |
| audit and final repro check | confirmed | Re-ran on main (fails); reduced the repro to the single test shown here. |
| this doc | reproduced; fix A validated | Gradle on main (on, off) fails, and so do the 10.6.0 and 9.5.0 jars; fix A passes the repro, 10 suites and check -x test; illustration probe. |
The diagram as a still image, generated with AI and checked against the code. It is under 10 MB, the limit for a GitHub attachment.
- PNG 477 KB
Finding record, Lucene and upstream sources, repro files and diagram notes1,581 words
- Registry title
- LatLonShapeDocValues and XYShapeDocValues getEncodedMinX/MinY/MaxX/MaxY return a tree node's box instead of the shape's bounding box after a relate() call that descends the shape's tree, because the getters read the reader's reusable scratch box
- Area
- core / document: shape doc values (ShapeDocValues.ShapeComparator bounding box behind LatLonShapeDocValues / XYShapeDocValues getEncodedMinX, getEncodedMinY, getEncodedMaxX, getEncodedMaxY)
- Priority
- Low, rank 431: After a relate() call that descends the shape's tree, the experimental shape doc-values bounding-box getters silently return a tree node's box, in every release since 9.5.0. Query results and getBoundingBox() are unaffected and only client code that reads these getters after relate() on the same instance is hit, so it is rare.
- Impact class
- minor
- Impact note
- Minor: the wrong values reach only client code reading the experimental getters after relate(). It hits only client code that builds a LatLonShapeDocValues or XYShapeDocValues through the public factories createLatLonShapeDocValues / createXYShapeDocValues(BytesRef) (released since 9.5.0, @lucene.experimental), calls relate(Component2D) with a query that crosses the shape's bounding box and reaches a non-root tree node (any shape with two or more tessellation components), and then reads getEncodedMinX/MinY/MaxX/MaxY on the same instance: the getters then silently return the last visited node's box, identically with -ea and -da [exec], until a later relate() decided at the root puts the header box back; relate() results, getBoundingBox() and every Lucene query are unaffected (BaseShapeDocValuesQuery builds a fresh instance per document and no Lucene code reads the getters after relate()), no setting and no scale are needed, and the one known downstream client (OpenSearch's GeoShapeDocValue) uses getBoundingBox() instead.
- Summary
- You build a LatLonShapeDocValues (or XYShapeDocValues) from a shape's doc-value bytes, call relate() with a query box, and then read getEncodedMinX/MinY/MaxX/MaxY, which are documented as 'the shape's bounding box'; you expect the same four values as before the relate() call. Instead, when the query box crosses the shape's box so that relate() walks down the shape's tree, the getters afterwards return the box of the last tree node visited: for the points (0,0), (10,10), (20,20) and a small box around (0,0), relate() correctly answers CELL_CROSSES_QUERY but getEncodedMaxX() then returns 0 instead of encodeLongitude(20) = 238609294, with no error. The getters read the reader's single reusable scratch box, which relate() overwrites for every node it visits, while getBoundingBox() of the same object, computed once in the constructor, stays correct.
- Introduced
- d7fd48c9502 on main ('LUCENE-10654: Add new ShapeDocValuesField for LatLonShape and XYShape (#1017)', Nick Knize; migrated issue #11690; branch_9x copy ddf0d0acf4e, PR #1064, first released in 9.4.0) [read: git blame attributes all four defect lines to it: the Reader's single scratch BBox (ShapeDocValues.java:577-585), readBBox()/resetBBox() returning bbox.reset(..), which returns this (:599-612, :663-670), ShapeComparator's 'this.boundingBox = dvReader.readBBox()' (:691, a reference, not a copy) and the recursive relate's per-node dvReader.resetBBox(..) (:836); git log -G'readBBox|resetBBox|boundingBox' on the file finds no other commit]. Client reachability came with e7253f112de (PR #11753, issue #11752, Navneet Verma, author date 2022-10-28; branch_9x copy a4ef70f0656, PR #11890), which added the public factories LatLonShape.createLatLonShapeDocValues(BytesRef) and XYShape.createXYShapeDocValues(BytesRef). Not a regression: the two later commits on ShapeDocValues.java (fad3108b27b, #11974, a Writer serialization flag; de3b294be44, formatting) do not touch these lines. (2022-08-09 (author and commit date of d7fd48c9502 and ddf0d0acf4e; 9.4.0 tagged 2022-09-30); reachable from client code since e7253f112de (author date 2022-10-28), first released in 9.5.0 (tagged 2023-01-30))
- First affected version
- 9.5.0 [exec] (the first release with the public factories createLatLonShapeDocValues / createXYShapeDocValues, so the first where client code can obtain an instance; the finder's repro fails there). The defective code first shipped in 9.4.0 [exec: a package-access variant fails on 9.4.0 and 9.4.2 and passes on 9.4.0 with the one-line fix shadowed], but 9.4.x exposes no instance to client code (both classes have protected constructors and no public method returns one; the finder's repro does not compile on 9.4.0 or 9.4.2); 9.3.0 has no shape doc values [exec: compile error].
- Affected versions
- Every release from 9.5.0 through 10.6.0 (released 2026-10-29, 4ae946e5a9b), 27 releases, and main 5adff78bb168 (12.0.0-SNAPSHOT); not fixed upstream. [exec] the finder's one-test repro fails with 'arrays first differed at element [1]; expected:<238609294> but was:<0>' (getEncodedMaxX) on main (-ea and -da, also in the re-run), 10.6.0 (-ea and -da), 10.0.0, 9.12.3 and 9.5.0 (-da); the independent reproduction's two-method test (a polyline, LatLon and XY) fails on main (-ea, -da), 10.6.0, 10.5.2, 9.12.3 and 9.5.0; the intended-behaviour check's XY polygon test fails on main (-ea, -da) and on the 10.6.0 jars. [read] the alias line, the scratch reset and the per-node resetBBox are present at every 9.x/10.x tag from 9.4.0 to 10.6.0 (follow-up tag sweep); ShapeDocValues.java is identical at 10.6.0 and main and unchanged on upstream main 1c2072f02843 (2026-11-02) since de3b294be44 (author date 2023-10-10). 9.4.0-9.4.2 contain the code but give client code no instance. Not new in 10.6. All runs JDK 25.0.3 on Linux aarch64.
- Status
- documented (brief, checks and a per-finding doc exist)
- Verification
- Confirmed by AI-run checks: an independent repro, a cross-model review that agreed, an intended-behaviour check, a prior-report search and a skeptical audit; not yet triaged by a person.
- Found by
- semi-formal: proof-obligation audit
- Upstream and prior reports
- none found (2026-11-02, on top of the finder's 45-query scripted prior-report search, the independent reproduction's and the intended-behaviour check's searches (the intended-behaviour check: 27 more queries) and the adjudication's re-check; rechecked the same day): a scripted prior-report search of 53 queries (24 GitHub issue and PR queries on apache/lucene, 3 commit queries, 2 downstream-repository queries, 20 queries over the dev@, java-user@, issues@ and commits@ lists, 4 JIRA-archive queries; 40 distinct hits, none the same bug), plus manual GitHub searches (class names, the getter names getEncodedMinX/MaxX/MinY/MaxY, readBBox, resetBBox, ShapeComparator, symptom phrases; open PRs), GitHub code search (no client caller of getEncoded* outside copies of Lucene's own sources), the comments of #1017, #11690, #11752, #11753, #11974, #1066, #13531 and #13743 and the review comments and reviews of #1017, the Elasticsearch, OpenSearch and Solr trackers, list and JIRA searches and 6 web queries: no report of the getters returning a sub-tree's box after relate(), by anyone. Same-day recheck: GitHub search on apache/lucene for 'ShapeDocValues' (7 items, all known: #12688, #12460, #11752, #11973, #11974, #11753, #1017), 'getEncodedMinX' (0), 'LatLonShapeDocValues bounding box' (0), 'XYShapeDocValues' (only #11753), open PRs 'ShapeDocValues' (only #12688, unrelated); none of the 32 apache/lucene PRs updated since 2026-10-31 touches a shape or geo file; the last upstream commit on ShapeDocValues.java is still de3b294be44 (2023-10-10), upstream main head 1c2072f02843. Related, not reports: #1017 / #11690 (LUCENE-10654, the introducing feature), #11752 / #11753 (made relate() and the factories public; no discussion of the getters), #11973 / #11974 (Writer serialization flag, another defect in the same file), #1066 / #1067 (test box computed from the uncleaned geometry).
- Lucene commit tested
- main
5adff78bb168(12.0.0-SNAPSHOT, 2026-11-02) through gradle in a scratch worktree, with assertions on and off, without and with the fix (the repro, 10 shape and shape doc-values suites, and:lucene:core:check -x test); the repro on the 10.6.0 and 9.5.0 release jars (fail); a public-API illustration probe on main with assertions on and off; fix B shadowed on main (the repro with assertions on and off,TestShapeDocValues7 of 7: pass). The hunt ran its tests on the 9.5.0, 9.12.3, 10.0.0, 10.5.2 and 10.6.0 release jars (all fail) and on 9.4.0 / 9.4.2 through package access (fail; no public factory there). Cross-model reviews ran on mainea57360912ed, whoseShapeDocValues.javais identical. Only Linux aarch64 with OpenJDK 25.0.3 was run.
- Finding record
lucene/core/src/java/org/apache/lucene/document/ShapeDocValues.java:49 (at 5adff78bb168) (@lucene.experimental), :104-123 (public getEncodedMinX/MinY/MaxX/MaxY, javadoc 'returns the min x value for the shape's bounding box', read shapeComparator.getMinX()..), :163-165 (public relate(Component2D), no javadoc), :577-585 (Reader keeps one scratch BBox), :599-612 (readBBox() and resetBBox() both return bbox.reset(..)), :663-670 (BBox.reset overwrites the fields and returns this), :680 and :691 (ShapeComparator 'private final SpatialQuery.EncodedRectangle boundingBox = dvReader.readBBox()': an alias of the scratch box, fix site), :679 and :681-683 (numberOfTerms, centroidX, centroidY and highestDimension held by value as final int and enum fields, assigned at :690 and :692-694), :702-716 (getMinX..getMaxY read it), :750 (relate re-reads the header box into the scratch box), :836 (dvReader.resetBBox(tMinX, tMaxX, tMinY, tMaxY) per visited non-root node; nothing restores it)lucene/core/src/java/org/apache/lucene/document/LatLonShapeDocValues.java:32 (at 5adff78bb168) (@lucene.experimental), :66-71 (computeBoundingBox builds getBoundingBox() from the four getters, in the constructor); XYShapeDocValues.java:67-72 (same); LatLonShape.java:350-352 and XYShape.java:280-282 (public factories); lucene/core/src/java/org/apache/lucene/geo/Component2D.java:27 (@lucene.internal); BaseShapeDocValuesQuery.java:78 (a fresh instance per document)https://github.com/apache/lucene/pull/1017, https://github.com/apache/lucene/issues/11690, https://github.com/apache/lucene/issues/11752, https://github.com/apache/lucene/pull/11753, https://github.com/apache/lucene/issues/11973, https://github.com/apache/lucene/pull/11974, https://github.com/apache/lucene/pull/1066, https://github.com/opensearch-project/OpenSearch/blob/main/server/src/main/java/org/opensearch/common/geo/GeoShapeDocValue.java, read 2026-11-02
- The finding in 30 seconds
lucene/core/src/java/org/apache/lucene/document/ShapeDocValues.java:578 (at 5adff78bb168) (scratch BBox), :599-612 (readBBox, resetBBox), :663-670 (BBox.reset returns this), :679-683 and :690-694 (header fields), :680 and :691 (boundingBox alias), :105-123 (public getters, javadoc), :702-716 (getMinX..getMaxY), :743-802 (relate entry, :750 header re-read), :826-831 (disjoint skip), :836 (per-node resetBBox), :80 and :92 (computeBoundingBox in the constructors)lucene/core/src/java/org/apache/lucene/document/LatLonShapeDocValues.java:66-71 (at 5adff78bb168) and XYShapeDocValues.java:67-72 (computeBoundingBox from the getters)
- Who is affected
lucene/core/src/java/org/apache/lucene/document/LatLonShape.java:350-352, (at 5adff78bb168) XYShape.java:280-282 (public factories); ShapeDocValues.java:163-165 (public relate); BaseShapeDocValuesQuery.java:78; LatLonShapeDocValuesQuery.java:42-43
- Why it looks unintended
lucene/core/src/java/org/apache/lucene/document/ShapeDocValues.java:105-123 (at 5adff78bb168) (javadoc), :163-165 (relate, no javadoc), :679-683 (header fields)lucene/core/src/test/org/apache/lucene/document/TestShapeDocValues.java:57-77 (at 5adff78bb168) (getBoundingBox checks only)https://github.com/apache/lucene/issues/11752 (title, read 2026-11-03); https://github.com/apache/lucene/pull/11753; https://github.com/apache/lucene/pull/1017
- Suggested fix
lucene/core/src/java/org/apache/lucene/document/SpatialQuery.java:803-817 (at 5adff78bb168) (EncodedRectangle: public static class, protected constructor, same package as ShapeDocValues)
- TestShapeDocValuesBBoxAfterRelate.java (lucene/core, public API):
repro/LFV-1130/TestShapeDocValuesBBoxAfterRelate.java(36 lines)
A light diagram (PNG), drawn from a probe run on main 5adff78bb168 with assertions on and off.
assets/LFV-1130/bug.png(477 KB)