BBoxStrategy(the spatial-extras module) indexes one rectangle per document.makeOverlapRatioValueSource(queryBox, p)turns the overlap between that rectangle and a query rectangle into a relevance score.- The score mixes two shares of the overlap: queryRatio, the part of the query box it covers, and targetRatio, the part of the document box it covers, weighted p and 1 - p.
IndexSearcher.explainorDoubleValuesSource.explainreturns a tree that shows how a score was computed: one node per term, with the values that went into it as children.
queryFactor (0.0625) the ratio shows the document's 1.0 instead of the query's 0.125; the score 0.5625 is right. Select the still to open it at full size. Generated with AI, checked against the code.- Index one document whose box is x 10..30, y 0..10 (area 200) with
BBoxStrategy, and score it withmakeOverlapRatioValueSourcefor the query box x 0..40, y -20..20 (area 1,600) and weight 0.5. The document box lies inside the query box, so the overlap is 200. - The score is queryFactor + targetFactor: queryRatio = 200 / 1,600 = 0.125 times 0.5 gives 0.0625, and targetRatio = 200 / 200 = 1.0 times 0.5 gives 0.5. Lucene scores the document 0.5625, which is correct.
- You call
explain()and expect thequeryFactornode (0.0625) to show its ratio, 0.125, next to the query box's area, 1,600. - Instead it shows
1.0 = ratio: the document's ratio, the same number as undertargetFactor. The query's ratio appears nowhere in the tree. - Why: line 255 of
BBoxOverlapRatioValueSourcepassestargetRatiowherequeryRatiobelongs: the same line as the targetFactor one, written in the 2015 Explanation API rewrite (LUCENE-6446), where the code before it hadqueryRatio. 5.1.0 still showed the right number.
Anyone who reads explain() output for BBoxStrategy's overlap-ratio relevance, an opt-in, experimental feature of the spatial-extras module, for example while tuning spatial ranking; by code reading, Solr's BBoxField with score=overlapRatio builds the same class. Wrong whenever the two ratios differ, e.g. for boxes that overlap by a positive area and differ in area; scores, ranking and matching are correct.
Mechanism, terms and numbers533 words
BBoxStrategy.makeOverlapRatioValueSource(queryBox, p) scores a document by how much its box and the query box overlap: score = queryRatio x p + targetRatio x (1 - p), where queryRatio is the overlap divided by the query box's area and targetRatio the overlap divided by the document box's area, as the class javadoc defines it. The score is computed correctly, but the explanation puts targetRatio under the queryFactor node (BBoxOverlapRatioValueSource.java:255), next to the query box's area, so a match tree shows the document-side ratio twice and the query-side ratio never. For a 20 x 10 document box inside a 40 x 40 query box at p = 0.5 it prints 1.0 = ratio under 0.0625 = queryFactor, where queryRatio is 0.125; the score 0.5625 is right. The slip came from the mechanical Explanation API rewrite LUCENE-6446 in 2015: 5.1.0 shows queryRatio, every release from 5.2.0 to 10.6.0 and main shows targetRatio (113 releases executed: a 112-release sweep 4.10.0-10.5.2, plus 10.6.0). Changing the one token back to queryRatio fixes it and passes the whole spatial-extras test suite. Low priority: explain output only, in an opt-in, experimental relevance function.
- The class javadoc defines
queryRatio = intersectionArea / queryArea,queryFactor = queryRatio * queryTargetProportion,targetFactor = targetRatio * (1 - queryTargetProportion)andscore = queryFactor + targetFactor(BBoxOverlapRatioValueSource.java:39-48). score(Rectangle, AtomicReference<Explanation>)computes the two ratios (:208-230) and the score exactly that way (:236-238).- Only when a holder is passed (
exp != null) does it build the tree (:240-262). ThequeryFactornode isExplanation.match((float) queryFactor, "queryFactor", Explanation.match((float) targetRatio, "ratio"), Explanation.match((float) queryArea, "area of " + queryExtent ...))(:252-256): line 255 passestargetRatio. ThetargetFactornode (:257-261) passestargetRatiotoo, which is correct there. - The scoring path,
BBoxSimilarityValueSource.getValues, callsscore(rect, null)(BBoxSimilarityValueSource.java:75), so the tree is never built while scoring. Onlyexplain(:112-129) passes a holder;FunctionScoreQuery's weightexplainreaches it (FunctionScoreQuery.java:175-190).
score=0.5625
0.5625 = BBoxOverlapRatioValueSource: queryFactor + targetFactor
200.0 = IntersectionArea
20.0 = width
10.0 = height
0.5 = queryTargetProportion
0.0625 = queryFactor
1.0 = ratio <- targetRatio; queryRatio is 0.125
1600.0 = area of Rect(minX=0.0,maxX=40.0,minY=-20.0,maxY=20.0)
0.5 = targetFactor
1.0 = ratio
200.0 = area of Rect(minX=10.0,maxX=30.0,minY=0.0,maxY=10.0)(The <- comment is added; the rest is the program's output.) With the fix the node shows 0.125 = ratio and every other line is identical.
Neither ratio needs to be 1: for a partial overlap (query x 0..20, y 0..10; document x 10..40, y 5..15; p = 0.3) the tree shows 0.16666667 = ratio (targetRatio) under 0.075 = queryFactor, where queryRatio is 0.25. It is invisible only when the two ratios are equal: for boxes with an area and no minimum side length, both ratios divide the same overlap, so they are equal exactly when the overlap is zero (boxes that only touch: 0.0 = ratio under both factors) or the document box has the same area as the query box. With a minimum side length the padded areas decide, and point and line boxes use their own ratios (:208-230).
Exposure and likelihood247 words
- Who. Callers of
IndexSearcher.explain(for example on aFunctionScoreQuery) orDoubleValuesSource.explainon a source frommakeOverlapRatioValueSourceor the publicBBoxOverlapRatioValueSourceconstructor. They are told the wrong reason for a score: the query-side share of the overlap is never shown. - Default or opt-in. Opt-in:
BBoxStrategyandBBoxOverlapRatioValueSourceare@lucene.experimentalin the optional spatial-extras module, and explain is a diagnostic call. Everything else is default (SpatialContext.GEO,BBoxStrategy.newInstance, codec,IndexWriterConfig, searcher). One document is enough. - Scale. Every explained hit whose two ratios differ, which for boxes with an area (and no minimum side length) means every document box that overlaps the query box by a positive area and whose area differs from the query box's. In a random sample (including point and line boxes), 655 explained hits got a full tree (boxes that intersect, touching included) and 396 of them showed a wrong value.
- Downstream (code reading, not run). Solr's
BBoxField(score=overlapRatio) and its legacyBBoxStrategycopy construct this class, so Solr's debug explain output would show the same number. A code search for the class name in the elastic and opensearch-project organisations found no use. - Not affected. Scores, sort values, ranking, matching and the index: main and the fix gave identical ranked lists and score bits over 60 queries x 300 documents.
- Likelihood: plausible. A reader of the tree meets it at once, but few read it closely; no report of it was found, although it has been there since 5.2.0 (2015).
One JUnit test, public API only: index one box with BBoxStrategy, explain the overlap-ratio source for one query box, and assert that the ratio under queryFactor is queryRatio, 0.125. The boxes are the "within" case of the existing TestBBoxStrategy.testOverlapRatio, which checks only the score.
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.lucene.spatial.bbox;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.index.DirectoryReader;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.search.Explanation;
import org.apache.lucene.store.ByteBuffersDirectory;
import org.apache.lucene.store.Directory;
import org.apache.lucene.tests.util.LuceneTestCase;
import org.locationtech.spatial4j.context.SpatialContext;
public class TestBBoxOverlapRatioExplainQueryRatio extends LuceneTestCase {
public void testQueryFactorRatioIsQueryRatio() throws Exception {
SpatialContext ctx = SpatialContext.GEO;
BBoxStrategy strategy = BBoxStrategy.newInstance(ctx, "bbox");
try (Directory dir = new ByteBuffersDirectory()) {
try (IndexWriter writer = new IndexWriter(dir, new IndexWriterConfig())) {
Document doc = new Document();
for (Field f : strategy.createIndexableFields(ctx.getShapeFactory().rect(10, 30, 0, 10))) {
doc.add(f);
}
writer.addDocument(doc);
}
try (DirectoryReader reader = DirectoryReader.open(dir)) {
Explanation exp =
strategy
.makeOverlapRatioValueSource(ctx.getShapeFactory().rect(0, 40, -20, 20), 0.5)
.explain(reader.leaves().get(0), 0, Explanation.match(0f, "score"));
Explanation queryFactor = exp.getDetails()[1];
// queryFactor's "ratio" must be queryRatio 200/1600 = 0.125; main shows targetRatio 1.0
assertEquals(
queryFactor.getDescription() + " ratio",
0.125f,
queryFactor.getDetails()[0].getValue().floatValue(),
0f);
}
}
}
}
cd $LUCENE && ./gradlew :lucene:spatial-extras:test --tests org.apache.lucene.spatial.bbox.TestBBoxOverlapRatioExplainQueryRatio -Ptests.seed=DEADBEEF -Ptests.asserts=true
- Expected
- The
ratiounderqueryFactoris queryRatio = 200 / 1,600 = 0.125. - Observed
AssertionError: queryFactor ratio expected:<0.125> but was:<1.0>, with assertions on and off; 1.0 is targetRatio. With the fix the test passes.
More files, environment, full output262 words
One test class, one method, one document, single thread. The test body uses fixed boxes, a ByteBuffersDirectory, a plain IndexWriterConfig and no random() calls; LuceneTestCase still randomizes ambient settings such as the default codec, locale and time zone, which do not affect this arithmetic. The result does not depend on the seed. The recorded runs also passed --no-daemon --offline to gradle. The test passes the module's checkGoogleJavaFormat, ecjLintTest and forbiddenApisTest as is; a committer may prefer newDirectory() and newIndexWriterConfig(), which give the same result.
The boxes and the proportion are those of the "within" assertion in TestBBoxStrategy.testOverlapRatio (query shiftedRect(0, 40, -20, 20, ...), proportion 0.5, document shiftedRect(10, 30, 0, 10, ...), score (200d / 1600d * 0.5) + (0.5)), which calls score(rect, null) and never looks at the tree.
Regression test: this same test. It fails on stock main and passes with the fix.
| lucene | main 645d4fe14a20 (11.0.0-SNAPSHOT) and 10.6.0: fails; 5.2.0 to 10.5.2: affected (probe); 5.1.0 and earlier: correct; with the fix: passes |
|---|---|
| jdk | OpenJDK 25.0.3 |
| os | Linux aarch64 (the only platform run) |
cp $REPRO/TestBBoxOverlapRatioExplainQueryRatio.java $LUCENE/lucene/spatial-extras/src/test/org/apache/lucene/spatial/bbox/
cd $LUCENE && ./gradlew :lucene:spatial-extras:test --tests org.apache.lucene.spatial.bbox.TestBBoxOverlapRatioExplainQueryRatio -Ptests.seed=DEADBEEF -Ptests.asserts=true
# same failure with -Ptests.asserts=false
# gradle, main 645d4fe14a20, seed DEADBEEF, -Ptests.asserts=true
TestBBoxOverlapRatioExplainQueryRatio > testQueryFactorRatioIsQueryRatio FAILED
java.lang.AssertionError: queryFactor ratio expected:<0.125> but was:<1.0>
at __randomizedtesting.SeedInfo.seed([DEADBEEF:D42E575A05379FD6]:0)
at junit@4.13.2/org.junit.Assert.fail(Assert.java:89)
at junit@4.13.2/org.junit.Assert.failNotEquals(Assert.java:835)
at junit@4.13.2/org.junit.Assert.assertEquals(Assert.java:577)
at org.apache.lucene.spatial.bbox.TestBBoxOverlapRatioExplainQueryRatio.testQueryFactorRatioIsQueryRatio(TestBBoxOverlapRatioExplainQueryRatio.java:50)
1 test completed, 1 failed
BUILD FAILED
# -Ptests.asserts=false: the same AssertionError, BUILD FAILED
# release 10.6.0, -ea and -da: the same AssertionError
# with the fix, assertions on and off: BUILD SUCCESSFUL- The javadoc defines
queryFactor = queryRatio * queryTargetProportion;targetRatioplays no part in it. - The 2014 code showed
queryRatiohere; the 2015 change was an API-only rewrite that never discussed spatial code. - Lucene fixes wrong explain values as bugs, e.g. GITHUB#13691, an explain-only fix in SigmoidFunction (9.12.0).
The full argument205 words
- The contract.
BBoxOverlapRatioValueSource.java:39-48defines the formula and:236computes it.Explanation.matchdocuments its details as "sub explanations that contributed to this explanation" (Explanation.java:27-33). - The node is about the query side. Its other child is
area of <query box>; thetargetFactornode mirrors it withtargetRatioandarea of <document box>. - History. The original code (trunk
7e0aff8dabc, LUCENE-5714/LUCENE-5779, by dsmiley, 2014; backported to branch_4x asc608265684c, first released in 4.10.0) hade.addDetail(new Explanation((float)queryRatio, "ratio"))underqueryFactor. LUCENE-6446 (c6821e1a194, by jpountz, 2015) replaced it withExplanation.match((float)targetRatio, "ratio"), the same line as in thetargetFactorblock. Its issue (#7505) and its 16 comments discuss the API change, and none of them mentions spatial code, BBox or the ratios. - The tree was meant to be read. On LUCENE-7737 (#8788, 2017) dsmiley called this explanation "very very useful" when it was at risk of being lost. LUCENE-7741 (#8792,
fb24d451cc9, 2017-05-10) addedDoubleValuesSource.explainin response, so the explanation was kept when the class moved (the LUCENE-7737 commit2f2e00ffe23); it was never removed. The captured discussion does not examine the ratio values. - Precedent.
lucene/CHANGES.txtlists explain-only value errors under Bug Fixes: GITHUB#13691 (SigmoidFunction exponent, 9.12.0) and LUCENE-6042 (CustomScoreQuery explain, 5.0.0).
The devil's-advocate check123 words
The strongest case for "not a bug": both classes are @lucene.experimental; Explanation is an Expert diagnostic API; the child is only labelled "ratio", so it does not say which ratio; the bbox tree is not a strict expression tree (the proportion sits under IntersectionArea); no test checks the explanation; the score is right; and no complaint was found in eleven years.
Why it fails: experimental marks API stability, not a licence for wrong output. The javadoc names the quantity, the sibling node shows the pattern, the 2014 code showed queryRatio, and the change came from a refactoring that set out to change no displayed value. The rest argues for Low priority, which is how it is classed, not for intent.
Recommended Show queryRatio under queryFactor (one token)
Pass queryRatio instead of targetRatio to the ratio child of the queryFactor node (BBoxOverlapRatioValueSource.java:255, one token). Scores and the tree's shape do not change. The repro and the whole spatial-extras test suite (118 tests) pass, with assertions on and off.
Explanation.match(
(float) queryFactor,
"queryFactor",
- Explanation.match((float) targetRatio, "ratio"),
+ Explanation.match((float) queryRatio, "ratio"),
Explanation.match((float) queryArea, "area of " + queryExtent + minSideDesc)),All options and trade-offs3 options · 264 words
One fix was validated through gradle in a scratch worktree of 645d4fe14a20. Two optional clarity changes on top of it were not run.
Restores the 2014 behaviour of the node.
Explanation.match(
(float) queryFactor,
"queryFactor",
- Explanation.match((float) targetRatio, "ratio"),
+ Explanation.match((float) queryRatio, "ratio"),
Explanation.match((float) queryArea, "area of " + queryExtent + minSideDesc)),Trade-offs
None found: no change to scores, tree shape, descriptions or cost.
What the checks validated
- Scratch worktree of
645d4fe14a20, gradle--offline, OpenJDK 25.0.3, seed DEADBEEF. Stock: the repro fails with assertions on and off. - With the fix: the repro passes both ways; the whole
:lucene:spatial-extras:test(27 suites, 118 tests includingTestBBoxStrategy.testOverlapRatioand the repro; 5 skipped by the suites themselves, one@Ignoreand 4@Nightly) passes with 0 failures with assertions on and off;checkGoogleJavaFormat,ecjLintMain,ecjLintTest,forbiddenApisMainandforbiddenApisTestpass. - Earlier, the finder's, the independent reproduction's, the cross-model review's and the intended-behaviour check's tests and the history probe passed with the same change.
Add queryTargetProportion (and 1 - queryTargetProportion) as a child of queryFactor (and targetFactor), so each node multiplies out.
Trade-offs
Clearer, but changes the tree's shape, which someone may parse or snapshot, and is more than the bug needs. A possible follow-up.
Describe the children as queryRatio and targetRatio instead of the generic ratio.
Trade-offs
Would make this kind of slip visible, but changes the text of explain output for no functional gain.
Solr's BBoxField builds this class for score=overlapRatio (by code reading, not run), so Solr's debug explain output for such queries would show the same wrong ratio.
Code reading: while an AI finder wrote an exact reference of the overlap-ratio score formula for a bounded-exhaustive score check, it noticed targetRatio in the queryFactor node. A probe printed the tree (score right, ratio wrong), and a public-API JUnit failed on the stock jars.
The method in full109 words
Method: code reading (the hunt's non-formal tier).
- The spatial-extras hunt unit wrote an exact reference of
BBoxOverlapRatioValueSource's score formula for a bounded-exhaustive check of scores. That check compares scores only, so it did not witness this bug. - Reading the explain block next to it, the finder saw
targetRatiounderqueryFactor. A probe printed1.0 = ratiounder0.0625 = queryFactorwith the correct score 0.5625, and a public-API JUnit failed on main (-eaand-da), 10.5.2 and 9.12.3. - Then cross-model reviews (OpenAI GPT-5.6, reading only), an independent reproducer, a devil's-advocate check, an adjudication, history, prior-report and security steps, an audit and a repro minimization followed.
No prior report found (2026-10-26: 245 logged queries over GitHub, the JIRA archive, the Lucene and Solr lists, downstream trackers and the web, plus the files of all 411 open PRs; re-checked 2026-11-01). The closest items are the 2015 change that introduced it and 2017 work that valued this explanation.
Searches2 searches
Verdict new: no issue, PR, JIRA entry, list thread, commit or web page describes this, by anyone. Context, not reports: #7505 (LUCENE-6446, introduced it; never mentions spatial code); #6776 and #6841 (2014, created the class with queryRatio here); #8788 and #8792 (2017, the explanation called "very very useful"; the discussion does not examine the numbers); SOLR-6183 (Solr's score=overlapRatio); apache/solr#4145 (open PR to remove the BBox field type). No open Lucene PR touches the file.
| Where | When | How | Found |
|---|---|---|---|
| GitHub issues, PRs and commits (apache/lucene incl. the migrated JIRA archive, apache/solr, Elasticsearch, OpenSearch, all of GitHub), GitHub code search, JIRA LUCENE and SOLR, the dev@, java-user@, issues@, commits@ and Solr lists, the web, Lucene.NET | 2026-10-26 | 245 logged queries: two scripted prior-report search runs, 59 read-only GitHub API searches, 6 code searches, 14 JIRA queries, 37 list searches, 20 web searches; the changed files of all 411 open PRs and of the 28 PRs among 229 spatial-labelled items | no report; 18 hits read, none the same bug |
| apache/lucene issues and PRs; upstream main | 2026-11-01 | read-only searches ("BBoxOverlapRatioValueSource", "queryRatio targetRatio", "bbox explain" and "spatial explain" updated since 2026-10-26) and the file on upstream main | nothing new; upstream main 2f424fd428da still shows targetRatio |
Introduced by LUCENE-6446 "Simplified Explanation API" (c6821e1a194 / 3b90a63a027, 2015-04-21), first released in 5.2.0. All 113 releases that ship the class were run: in the sweep of the 112 releases up to 10.5.2, 4.10.0 to 5.1.0 are correct and every release from 5.2.0 to 10.5.2 is affected; 10.6.0, run separately, and main fail too. Not fixed upstream.
Versions tested6 rows
Introduced by LUCENE-6446 "Simplified Explanation API" (#7505, by jpountz, no pull request): branch_5x 3b90a63a027 (shipped first, in 5.2.0) and trunk c6821e1a194 (ancestor of 6.x-10.x and main), both 2015-04-21. The rewrite replaced new Explanation((float)queryRatio, "ratio") with Explanation.match((float)targetRatio, "ratio"). git log -L on the block lists only 7e0aff8dabc (2014, with queryRatio), c6821e1a194 and the 2021 reformat 2cbf261032d (token unchanged). Not fixed: upstream main 2f424fd428da (2026-10-30) still shows targetRatio.
| Version or commit | Verdict | Result |
|---|---|---|
main 645d4fe14a20 | affected | minimal test fails through gradle with assertions on and off (expected:<0.125> but was:<1.0>) |
| 10.6.0 | affected | minimal test fails with -ea and -da |
| 10.5.2, 9.12.3 | affected | the finder's JUnit (same boxes) and the independent reproduction's partial-overlap JUnit fail |
| 5.2.0 to 10.5.2 (105 releases) | sweep | a public-API probe shows targetRatio under queryFactor on every one; 5.2.0 also with -da |
| 4.10.0 to 5.1.0 (7 releases) | sweep | the probe shows queryRatio; 5.1.0 also with -da |
| main + one-token fix | not affected | minimal test and the whole spatial-extras suite pass |
Reproduced by the finder, an independent reproducer (with a partial overlap and through IndexSearcher.explain), a devil's-advocate check, the adjudication, an audit, a repro minimization and this doc's gradle runs. Cross-model reviews (OpenAI GPT-5.6) could only read code; one gave a procedural "not real".
Checks and verdicts294 words
Every reproduction ran on stock builds; the fix checks shadowed or patched only BBoxOverlapRatioValueSource. The cross-model reviewers executed nothing themselves; the repro was re-run for each review and failed as claimed.
| Check | Verdict | What it found |
|---|---|---|
| Finder (spatial-extras unit) | candidate | Probe printed 1.0 = ratio under 0.0625 = queryFactor; public-API JUnit fails on main (-ea, -da), 10.5.2 and 9.12.3; passes with the fix. |
| Cross-model refute review (OpenAI GPT-5.6, reading only) | not real, low (procedural) | Could not run code, so its verdict is procedural; on reading it agreed with the root cause and rated the intended argument weak. |
| Independent reproducer | real | Own 3-test JUnit, partial overlap (0.25 vs 1/6): 2 of 3 fail on main (-ea, -da), 10.5.2 and 9.12.3; the control passes; 3 of 3 pass with the fix. |
| Cross-model review, intended lens (OpenAI GPT-5.6, reading only) | real, high | Rated the intended argument moderate, defeated by the javadoc and the sibling block; a separate JUnit fails on main and 9.12.3 and passes with the fix. |
| Devil's advocate | real, high | Built the strongest "intended" case and showed why it fails; end-to-end IndexSearcher.explain test fails on main (-ea, -da). |
| Adjudication | confirmed, high | Re-ran the independent reproduction's JUnit (-ea, -da); 3 of 3 pass with the fix. |
| Audit | confirmed | Re-ran the independent reproduction's JUnit on main with -da: 2 of 3 fail as claimed. |
| Repro minimization | verified | One test, one assertion; fails on 645d4fe14a20 with -ea and -da; passes with the fix. |
| This doc's runs | confirmed | Gradle in a scratch worktree: repro fails stock, passes fixed; whole module and lint pass with the fix; 10.6.0 fails. |
The diagram as a still image, generated with AI and checked against the code. It is under 10 MB, the limit for a GitHub attachment.
- PNG 523 KB
Finding record, Lucene and upstream sources, repro files and diagram notes1,523 words
- Registry title
- BBoxStrategy's overlap-ratio explanation (BBoxOverlapRatioValueSource) shows the target ratio under 'queryFactor' instead of the query ratio, so explain() misreports how the score was computed; the score itself is correct
- Area
- spatial-extras / BBoxStrategy overlap-ratio relevance (BBoxOverlapRatioValueSource.score: explanation tree)
- Priority
- Low, rank 1041: In this explain-only defect the overlap-ratio score is correct but the explain tree shows the wrong ratio.
- Impact class
- minor
- Impact note
- Anyone who calls IndexSearcher.explain (for example on a FunctionScoreQuery) or DoubleValuesSource.explain on BBoxStrategy's overlap-ratio relevance (makeOverlapRatioValueSource or the public BBoxOverlapRatioValueSource constructor, @lucene.experimental, in the opt-in spatial-extras module; Solr's BBoxField builds the same class for score=overlapRatio, by a read-only code search, so its debug output would show the same, not executed) sees the target-side ratio twice and the query-side ratio not at all for every explained document whose two ratios differ, which for boxes with an area and no minimum side length means every document box that overlaps the query box by a positive area and whose area differs from the query box's (396 of the 655 intersecting pairs, touching included, in a random sample), with default BBoxStrategy, codec, IndexWriterConfig and searcher, -ea and -da, one document, in every release since 5.2.0; scores, sort values, ranking, matching and the index are unaffected and nothing throws.
- Summary
- You score documents with BBoxStrategy.makeOverlapRatioValueSource and call explain, and you expect the tree to show the terms of the documented formula, queryFactor = queryRatio * queryTargetProportion and targetFactor = targetRatio * (1 - queryTargetProportion). Instead the 'ratio' child of the 'queryFactor' node, printed next to 'area of <query box>', is targetRatio, the same number as under 'targetFactor': for the query box x[0,20] y[0,10] and the document box x[10,40] y[5,15] at proportion 0.3 it shows 0.16666667 (50/300) where queryRatio is 0.25 (50/200). The score (0.19166666) and the queryFactor value (0.075) are right; one token on line 255 is wrong, a slip from the 2015 LUCENE-6446 Explanation API rewrite, and 5.1.0 still showed queryRatio.
- Introduced
- 3b90a63a027a3b62aeb7b3504dacc4447880de4f (branch_5x, svn r1675114, 2015-04-21 13:23 UTC, the copy that shipped first, in 5.2.0) and its trunk original c6821e1a1941e9ed6cbe35a3cb5be831bb6cfe1a (svn r1675109, 12:55 UTC, an ancestor of every 6.x-10.x release and main), 'LUCENE-6446: Simplified Explanation API', by jpountz; no pull request (SVN + JIRA era; LUCENE-6446 is apache/lucene#7505). This mechanical rewrite of every Explanation call site for the immutable API replaced e.addDetail(new Explanation((float)queryRatio, "ratio")) under 'queryFactor' with Explanation.match((float)targetRatio, "ratio"), the same line as in the targetFactor block; the issue and its 16 comments state API goals only and never mention spatial code. [read: git show of both commits; git log -L on main's explain block lists only 7e0aff8dabc (added with queryRatio), c6821e1a194 (queryRatio -> targetRatio) and the LUCENE-9570 reformat 2cbf261032d (token unchanged); git log -S; git blame -w ignoring the reformat; exec: 5.1.0 passes and 5.2.0 fails, -ea and -da]. A regression: the class's first version, 7e0aff8dabc4 (LUCENE-5714/5779, by dsmiley, 2014-07-08; first shipped in 4.10.0 through the branch_4x backport), showed queryRatio (2015-04-21 (trunk c6821e1a194 and branch_5x 3b90a63a027, the same day))
- First affected version
- 5.2.0 [exec] (tag 2015-06-06; CHANGES.txt lists LUCENE-6446 under 5.2.0; fails with -ea and -da); the last unaffected release is 5.1.0 [exec, -ea and -da]; 4.10.0-4.10.4 and 5.0.0 are also correct [exec]; no release before 4.10.0 has the class
- Affected versions
- Every release from 5.2.0 to 10.6.0 and main ea57360912ed; not fixed upstream. [exec] a public-API main() probe (four API spellings of one case: queryRatio 0.25, targetRatio 1/6) on all 112 Maven Central releases before 10.6.0 that ship the class (lucene-spatial 4.10.0-5.5.5, lucene-spatial-extras 6.0.0-10.5.2), -ea: 105 fail, every release from 5.2.0 to 10.5.2, showing 0.16666667 under queryFactor; 7 pass (4.10.0-4.10.4, 5.0.0, 5.1.0); 5.1.0 (pass) and 5.2.0 (fail) also with -da; main fails with -ea and -da and passes with the one-token fix shadowed; the finder's JUnit fails on main (-ea and -da), 10.5.2 and 9.12.3, the independent reproduction's JUnit on main (-ea and -da), 10.5.2 and 9.12.3, and a separate JUnit on main (-ea and -da) and 9.12.3. [read] source scan of the 103 release tags in the history clone that contain the class: queryRatio in the 7 from 4.10.0 to 5.1.0, targetRatio in the 96 from 5.2.0 to 10.5.2 (8.8.2 to 8.11.4 have no tag there and were checked by execution only); upstream main e1c4184cb983, branch_10x, branch_10_6, branch_9_12 and branch_9_11 still show targetRatio (read-only, 2026-10-26); 10.6.0 (released 2026-10-29) is affected [exec 2026-11-01: the minimal test fails on the 10.6.0 release jars (SHA-1 checked against Maven Central) with -ea and -da]. All runs with JDK 25.0.3 on aarch64, not the JDKs the old releases targeted (a pure-arithmetic path)
- Status
- documented (brief, checks and a per-finding doc exist)
- Verification
- Confirmed by AI-run checks: an independent repro, a cross-model review that could only read the code (its low-confidence “not real” is procedural), an intended-behaviour check, a prior-report search and a skeptical audit; not yet triaged by a person.
- Found by
- non-formal: code reading
- Upstream and prior reports
- none found (2026-10-26, read-only, verdict new: 245 logged queries: a scripted prior-report search in two runs (40 GitHub issue/PR, 6 commit, 4 downstream, 45 mailing-list over dev@, java-user@, issues@, commits@ and solr-user@, 8 JIRA), 59 read-only GitHub API searches (apache/lucene issues, PRs and commits, apache/solr, Elasticsearch, OpenSearch and all of GitHub), 6 GitHub code searches, 14 JIRA queries over LUCENE and SOLR, 37 direct mailing-list searches including the Solr lists, 20 web searches, the changed files of all 411 open apache/lucene PRs, all 229 spatial-labelled issues and PRs with the files of their 28 PRs, upstream branch history, Lucene.NET's port and GitHub Discussions; plus the finder's, the independent reproduction's and the intended-behaviour check's earlier searches, the intended-behaviour check's scripted prior-report search with 26 queries and 0 high hits); 18 hits read, none the same bug. Not fixed upstream: the file is unchanged since the 2021 LUCENE-9570 reformat (2cbf261032dc) on main e1c4184cb983, branch_10x and branch_10_6, no open PR touches it (the only open PR in the bbox package, #12295, changes BBoxStrategy.java casts only), and no GitHub-indexed copy has a corrected line. Related, not reports of this bug: #7505 (LUCENE-6446, the change that introduced it; its comments never mention spatial code), #6776 (LUCENE-5714) and #6841 (LUCENE-5779) (2014, created the class with queryRatio in this node), #8788 (LUCENE-7737, 2017: when the class moved to DoubleValuesSource, dsmiley called this explanation 'very very useful'; the captured discussion does not examine the ratio values) and #8792 (LUCENE-7741, added DoubleValuesSource.explain in response, so the explanation was kept), #8150 (unrelated, a 'BBoxStrategy explain' search hit), SOLR-6183 (exposes the class as BBoxField score=overlapRatio), apache/solr#4145 (SOLR-18111, an open PR to remove the deprecated BBox field type).
- Lucene commit tested
- main
645d4fe14a20(2026-11-01): the minimal test fails through gradle in a scratch worktree with assertions on and off (expected:<0.125> but was:<1.0>); with the one-token fix it passes, the whole:lucene:spatial-extras:testpasses (27 suites, 118 tests, 0 failures) both ways, and the module's format, ECJ and forbidden-apis checks pass. The minimal test also fails on 10.6.0 (-eaand-da). Earlier runs used a probe on the 112 releases up to 10.5.2 that ship the class (4.10.0-5.1.0 pass, 5.2.0-10.5.2 fail) and the finder's and the independent reproduction's tests on 10.5.2 and 9.12.3; with 10.6.0, the only later release, all 113 releases that ship the class were executed.
- Finding record
lucene/spatial-extras/src/java/org/apache/lucene/spatial/bbox/BBoxOverlapRatioValueSource.java:24-48 (class javadoc: queryRatio = intersectionArea / queryArea at :39-40, queryFactor = queryRatio * queryTargetProportion at :43-44, targetFactor at :45-46, score at :47-48), :67 (@lucene.experimental), :236-238 (queryFactor, targetFactor, score), :240-262 (explanation, built only when exp != null), :252-256 ('queryFactor' node; :255 Explanation.match((float) targetRatio, "ratio")), :257-261 ('targetFactor' node, targetRatio, correct)lucene/spatial-extras/src/java/org/apache/lucene/spatial/bbox/BBoxSimilarityValueSource.java:75 (getValues scores with score(rect, null)) and :113-129 (explain passes an explanation holder)lucene/spatial-extras/src/java/org/apache/lucene/spatial/bbox/BBoxStrategy.java:228-236 (makeOverlapRatioValueSource) and :71 (@lucene.experimental)lucene/core/src/java/org/apache/lucene/search/Explanation.java:24 ('Expert: Describes the score computation') and :27-33 (Explanation.match: details are 'sub explanations that contributed to this explanation')lucene/queries/src/java/org/apache/lucene/queries/function/FunctionScoreQuery.java:175-190 (weight explain calls valueSource.explain)
- The finding in 30 seconds
a Lucene main checkout (645d4fe14a20) lucene/spatial-extras/src/java/org/apache/lucene/spatial/bbox/BBoxOverlapRatioValueSource.java:39-48, 208-230, 236-238, 240-262 (queryFactor node :252-256, the token :255; targetFactor node :257-261)a Lucene main checkout (645d4fe14a20) lucene/spatial-extras/src/java/org/apache/lucene/spatial/bbox/BBoxSimilarityValueSource.java:75 (getValues: score(rect, null)), :112-129 (explain)a Lucene main checkout (645d4fe14a20) lucene/queries/src/java/org/apache/lucene/queries/function/FunctionScoreQuery.java:175-190
- Who is affected
a Lucene main checkout (645d4fe14a20) lucene/spatial-extras/src/java/org/apache/lucene/spatial/bbox/BBoxOverlapRatioValueSource.java:67 (@lucene.experimental); a Lucene main checkout (645d4fe14a20) lucene/spatial-extras/src/java/org/apache/lucene/spatial/bbox/BBoxStrategy.java:71 (@lucene.experimental), :228-236a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/search/Explanation.java:24 ('Expert: Describes the score computation')
- Minimal repro
a Lucene main checkout (645d4fe14a20) lucene/spatial-extras/src/test/org/apache/lucene/spatial/bbox/TestBBoxStrategy.java:227-300 (testOverlapRatio; the "within" case :239-253)
- Why it looks unintended
a Lucene main checkout (645d4fe14a20) lucene/spatial-extras/src/java/org/apache/lucene/spatial/bbox/BBoxOverlapRatioValueSource.java:39-48, 236, 252-261a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/search/Explanation.java:27-33a Lucene main checkout (645d4fe14a20) lucene/CHANGES.txt:2705 (GITHUB#13691, 9.12.0 Bug Fixes), :11550 (LUCENE-6042, 5.0.0 Bug Fixes)
- History
Maven Central maven-metadata.xml of org.apache.lucene:lucene-spatial-extras (lastUpdated 2026-10-29; 6.0.0 to 10.6.0, 10.6.0 the only version after 10.5.2) and org.apache.lucene:lucene-spatial (4.10.x and 5.x), read 2026-11-01
- Diagrams
a Lucene main checkout (645d4fe14a20) lucene/spatial-extras/src/java/org/apache/lucene/spatial/bbox/BBoxOverlapRatioValueSource.java:252-261
- TestBBoxOverlapRatioExplainQueryRatio.java (one test class):
repro/LFV-370/src/org/apache/lucene/spatial/bbox/TestBBoxOverlapRatioExplainQueryRatio.java(58 lines)
Made in parallel with this doc from the code at 645d4fe14a20 and the executed tree probe.
assets/LFV-370/bug.png(523 KB)
