All findings · Lucene finding · LFV-211 · functionscorequery-maxscore-uses-nonscoring-inner-bound

FunctionScoreQuery's max score is 0 when it wraps a TermQuery, so a BooleanQuery around it can skip the best hit

On Lucene main, FunctionScoreQuery(TermQuery, fromLongField(f)) over a skip-indexed field reports a block-max score of 0: it multiplies by the max score of its wrapped TermQuery, which it runs without scoring. Inside a BooleanQuery, search(q, 1) then returns doc 0 (score 4.69) instead of doc 4999 (score 100); unreleased, from #16431.

Summary

Priority
Medium, rank 102On main since #16431 FunctionScoreQuery's block-max bound multiplies by the wrapped TermQuery's no-score max of 0, so a BooleanQuery with a skip-indexed function clause silently loses its true top hits with ordinary values, the new feature's main use case, to fix before 11.0.
Impact
silent wrong results
Since
#16431 on main (unreleased)introduced 2026-08-22
Affected
main (11.0.0-SNAPSHOT) only; 10.x releases are correct
Status
documented
Found by
modelling the bound, an SMT check and a top-k differential

AI-assisted page: this finding was first signalled by a semi-formal check (an exact check of the real jar, or a modelling hypothesis that a tool then witnessed) and then checked with AI tools; the checks it went through, and what each found, are under Prior reports, history and checks. Not yet triaged by a human. Snapshot of 2026-11-06.

The finding in 30 seconds

FunctionScoreQuery(q, DoubleValuesSource.fromLongField(f)) scores each match of q by the value of field f. Since #16431 it also reports each block's maximum score so that top-k search can skip blocks. With a TermQuery as q and a skip index on f, that maximum is 0.

Summary card: Apache Lucene main, 5,000 docs, one segment, searcher.search(q, 1) with q = SHOULD FunctionScoreQuery(TermQuery body:a, fromLongField("popularity")) + SHOULD TermQuery body:b. 1: fromLongField needs no scores, so the wrapped TermQuery is weighted with COMPLETE_NO_SCORES; its dummy SimScorer returns 0f, so its max score is 0.0, while score() (popularity times boost) never reads it: doc 4999 scores 100.0. 2: getMaxScore returns innerMaxScore x valueMaxScore x boost = 0.0 x 100.0 x 1.0 = 0.0 for doc 4999's range. 3: after 1,000 hits the min competitive score is Math.nextUp(4.6875415) = 4.687542, and MaxScoreBulkScorer skips every range whose clause bounds sum to less. 4: doc 4999's range has the bound 0.0, so it is never scored. Expected (exhaustive search): doc 4999, 100.0. Actual: doc 0, 4.6875415, no error. Fix: innerMaxScore = 1 if the source needs no scores: doc 4999 first.
The repro step by step: the function query scores by popularity, its bound multiplies by the unscored TermQuery's 0, and after 1,000 hits the searcher skips every range whose bounds sum below 4.687542, including doc 4999's. Select the still to open it at full size. Generated with AI, checked against the code.
  1. Index 5,000 documents that all contain body:a; only doc 0 also contains body:b. A popularity field with a skip index (NumericDocValuesField.indexedField) is 1 everywhere and 100 on doc 4999.
  2. Search BooleanQuery(SHOULD FunctionScoreQuery(TermQuery(body:a), fromLongField("popularity")), SHOULD TermQuery(body:b)) with search(query, 1). You expect doc 4999 (score 100), which a search that scores every match returns.
  3. You get doc 0 (score 4.69 = popularity 1 + 3.69 for body:b). No error, with assertions on or off.
  4. Why: the function query tells the searcher that no document in a block scores more than (wrapped query's max score) × (max popularity). It runs the TermQuery without scores, so that max score is 0 and the claim is 0, although its documents score 1 to 100.
  5. After 1,000 hits only scores above 4.69 can compete, and a clause that claims 0 cannot reach that. From doc 4096 on the searcher scores nothing that only the function clause matches, so doc 4999 is never looked at.

Who is affected

Applications on unreleased main that put a skip index on a ranking field (NumericDocValuesField.indexedField), wrap a TermQuery or similar in FunctionScoreQuery with fromLongField, fromIntField, fromFloatField, fromDoubleField or a monotonic fromField, and use it as a scoring clause of a BooleanQuery. boostByValue, boostByQuery and a top-level FunctionScoreQuery are not affected.

Mechanism, terms and numbers687 words

FunctionScoreQuery(q, DoubleValuesSource.fromLongField(f)) scores each match of q by the value of f; its score() is the value times the boost and never reads q's score. Since #16431 (main, 2026-08-22) its scorer also reports a per-block maximum score so that top-k search can skip blocks, computed as innerMaxScore * valueMaxScore * boost. Because the field source does not need scores, the wrapped query was weighted with COMPLETE_NO_SCORES, and a TermQuery weighted that way reports a max score of 0. So on a field with a doc-values skip index every block with a finite bound claims 0, and a BooleanQuery that uses the function query as a scoring clause can stop scoring the documents only that clause makes competitive: in the repro, search(query, 1) returns doc 0 (4.6875415) instead of doc 4999 (100). No released version is affected.

Terms

  • Block-max pruning. For top-k search, each scoring clause reports, for a range of documents, the highest score any of them can have (Scorer#getMaxScore). Once the collector knows the k-th best score so far, the searcher skips ranges whose bounds cannot beat it, without scoring their documents.
  • Skip index. NumericDocValuesField.indexedField(name, value) adds a doc-values skip index: the minimum and maximum value per interval of documents. #16431 uses it to bound FunctionScoreQuery's scores per block.
  • Unscored wrapped query. FunctionScoreQuery needs its wrapped query only to pick the matching documents when the value source does not read scores, so it creates the wrapped query's weight with ScoreMode.COMPLETE_NO_SCORES ("scores won't be available").

The bug

  1. createWeight weights the wrapped query with COMPLETE_NO_SCORES unless the caller and the source both need scores (FunctionScoreQuery.java:109-117). Every field source (fromLongField, fromIntField, fromFloatField, fromDoubleField, fromField) does not need scores.
  2. A TermQuery weighted that way gets a dummy SimScorer that returns 0f, "not expected to be called since scores are not needed" (TermQuery.java:77-91). Its max score is 0.0.
  3. score() returns (float) (value * boost) for a non-negative value, else 0 (FunctionScoreQuery.java:243-252). The wrapped score is never read.
  4. getMaxScore(upTo) returns innerMaxScore * valueMaxScore * boost with innerMaxScore = in.getMaxScore(upTo) (:262-269). valueMaxScore is the skip index's maximum for the range (1.0 for docs 0-4998, 100.0 for doc 4999 in the repro), so the bound is 0.0 for both ranges, i.e. wherever the skip index gives a finite bound (a range past the current interval gets +Infinity). Without a skip index the value bound is +Infinity and the method returns +Infinity, which is why the skip index is part of the trigger.
  5. search(query, n) asks for TOP_SCORES from the start and counts the first 1,000 hits exactly; after that it raises the minimum competitive score to just above the lowest score in its top-n queue, the best score so far for n = 1 (IndexSearcher.java:100, TopScoreDocCollector.java:58-59, :154-170): 4.687542 after hit 1,001. MaxScoreBulkScorer then treats every clause whose summed bounds stay below that as non-essential and only iterates essential clauses (MaxScoreBulkScorer.java:494-510). From doc 4096 on, the function clause claims 0.0 and body:b, which has no documents left there, counts as 0 (:441-442); the sum, 0.0, is below 4.687542, so nothing in docs 4096-4999 is scored. The 4096 is the bulk scorer's inner window (INNER_WINDOW_SIZE = 1 << 12, :28): docs 0-4095 had been filled before the threshold was reached.
  6. A top-level FunctionScoreQuery alone is not pruned: its FilterScorer inherits the no-op setMinCompetitiveScore (Scorable.java:45-54). Its exposed getMaxScore is still 0.

The numbers

An instrumented run of the repro index on main (assertions on and off) shows: doc 0 scores 1.0 + 3.6875415 = 4.6875415, docs 1-4998 score 1.0, doc 4999 scores 100.0; the function scorer's getMaxScore is 0.0 for docs 0-4998 and for doc 4999; setMinCompetitiveScore(4.687542) after 1,001 collected hits; collection stops after 4,096 documents; search(query, 1) returns doc 0, and search(query, 3) docs 0, 1 and 2, all with total hits "4096+". With the finder's fix the bounds are 1.0 and 100.0 and search(query, 1) returns doc 4999 while still counting only "4097+" hits, so pruning still skips docs 4096-4998 (not benchmarked).

Exposure and likelihood351 words
  • Who is exposed. All of: unreleased main (11.0.0-SNAPSHOT) since #16431; a numeric doc-values field with a skip index; a score-independent field source with a finite bound; a wrapped query whose unscored max score is below 1; and the function query as a scoring clause of a BooleanQuery under the default search(query, n).
  • Wrapped queries. A TermQuery reports 0. The finder's randomized differential also lost top hits with a ConstantScoreQuery and with BooleanQuery disjunctions and conjunctions of terms as the wrapped query. MatchAllDocsQuery reports 1 and is not affected.
  • Compositions. A SHOULD clause next to another clause failed in every executed index shape. A MUST clause next to another scoring MUST clause failed in two independent tests, but whether it fails depends on how scoring windows line up with skip-index intervals.
  • Default or opt-in. The search side is all default (IndexSearcher, BM25, search(query, n)). The repro opts into a skip index on the value field, a schema-time choice, and forces one segment (forceMerge(1)) to keep the index shape fixed. The skip index is exactly what #16431's speed-up relies on (CHANGES.txt: "Block-Max WAND dynamic score skipping in FunctionScoreQuery and DoubleValuesSource using DocValuesSkipper"), so the exposed applications are those that adopt the new feature as advertised and also use the query shape above.
  • Scale. A few thousand documents suffice (5,000 in the repro; 8,203 and 10,000 in the reviewers' tests); the exposed bound is 0 at any size for every finite skip-index bound. The values are ordinary non-negative integers.
  • Not affected. A caller that needs no scores (then createWeight returns the wrapped weight itself, FunctionScoreQuery.java:118-120); boostByValue and boostByQuery (their source needs scores, so the bound stays +Infinity); a top-level FunctionScoreQuery alone (right hits, wrong exposed bound); fields without a skip index; every released version.
  • What the damage looks like. Silent in the plain repro (only a test that wraps the inner query itself in AssertingQuery fails an assertion): the returned hits are real matches with correct scores, but the best ones are missing, and the total hit count is the usual lower bound ("4096+").

Minimal repro

The test shows that BooleanQuery(SHOULD FunctionScoreQuery(TermQuery(body:a), fromLongField("popularity")), SHOULD TermQuery(body:b)) over a skip-indexed popularity field (NumericDocValuesField.indexedField) returns the wrong top hit from the default IndexSearcher.search(query, 1). The cause is that the function clause's block-max bound is the wrapped no-score TermQuery's max score of 0 times the value bound.

TestFunctionScoreQueryBooleanTopHit.java (JUnit test)71 lines
/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.apache.lucene.queries.function;

import java.io.IOException;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.document.NumericDocValuesField;
import org.apache.lucene.document.StringField;
import org.apache.lucene.index.DirectoryReader;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.index.Term;
import org.apache.lucene.search.BooleanClause.Occur;
import org.apache.lucene.search.BooleanQuery;
import org.apache.lucene.search.DoubleValuesSource;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.ScoreDoc;
import org.apache.lucene.search.TermQuery;
import org.apache.lucene.store.ByteBuffersDirectory;
import org.apache.lucene.store.Directory;
import org.apache.lucene.tests.util.LuceneTestCase;
import org.apache.lucene.tests.util.TestUtil;

public class TestFunctionScoreQueryBooleanTopHit extends LuceneTestCase {

  public void testTopHitWithSkipIndexedValueField() throws IOException {
    try (Directory dir = new ByteBuffersDirectory();
        IndexWriter w =
            new IndexWriter(dir, new IndexWriterConfig().setCodec(TestUtil.getDefaultCodec()))) {
      for (int i = 0; i < 4097; i++) {
        Document doc = new Document();
        doc.add(new StringField("body", "a", Field.Store.NO));
        if (i == 0) {
          doc.add(new StringField("body", "b", Field.Store.NO));
        }
        doc.add(NumericDocValuesField.indexedField("popularity", i == 4096 ? 100 : 1));
        w.addDocument(doc);
      }
      try (DirectoryReader reader = DirectoryReader.open(w)) {
        Query query =
            new BooleanQuery.Builder()
                .add(
                    new FunctionScoreQuery(
                        new TermQuery(new Term("body", "a")),
                        DoubleValuesSource.fromLongField("popularity")),
                    Occur.SHOULD)
                .add(new TermQuery(new Term("body", "b")), Occur.SHOULD)
                .build();
        ScoreDoc top = new IndexSearcher(reader).search(query, 1).scoreDocs[0];
        // expected doc 4096 (score 100 = its popularity); main returns doc 0 (score 1 + BM25 of b)
        assertEquals("top hit (returned score " + top.score + ")", 4096, top.doc);
      }
    }
  }
}
Command
# in a copy of this finding's repro folder (repro/LFV-211/, which holds src/)
# Lucene checkout at the tested commit, built with: ./gradlew --no-daemon assemble   (JDK 25)
LUCENE=/path/to/lucene; V=12.0.0-SNAPSHOT
CP=$LUCENE/lucene/core/build/libs/lucene-core-$V.jar
CP=$CP:$LUCENE/lucene/codecs/build/libs/lucene-codecs-$V.jar
CP=$CP:$LUCENE/lucene/test-framework/build/libs/lucene-test-framework-$V.jar
CP=$CP:$LUCENE/lucene/queries/build/libs/lucene-queries-$V.jar
CP=$CP:randomizedtesting-runner-2.9.2.jar:junit-4.13.2.jar:hamcrest-3.0.jar   # test-framework dependencies (Maven Central)
CP=$CP:junit-platform-console-standalone-6.1.3.jar   # JUnit Platform console launcher (Maven Central)

javac -d classes -cp "$CP" $(find src -name '*.java')

# run 1: assertions on, seed DEADBEEF
java -ea -esa --add-modules jdk.incubator.vector --enable-native-access=ALL-UNNAMED -Dtests.seed=DEADBEEF \
     -cp "classes:$CP" org.junit.platform.console.ConsoleLauncher \
     execute --select-class org.apache.lucene.queries.function.TestFunctionScoreQueryBooleanTopHit
Expected vs actual
Expected: doc 4096 (popularity 100, score 100). Actual on main with -ea and with -da: doc 0 (score 4.597 = 1 + BM25 of body:b), expected:<4096> but was:<0>.

All files of this repro: repro/LFV-211/

More files, environment, full output38 words

Environment

luceneLucene main e1faeba08bde (2026-11-02)
jdk25.0.3+9-LTS

Observed output

# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
    => java.lang.AssertionError: top hit (returned score 4.5970817) expected:<4096> but was:<0>
       org.junit.Assert.fail(Assert.java:89)
       org.junit.Assert.failNotEquals(Assert.java:835)
       org.junit.Assert.assertEquals(Assert.java:647)
       org.apache.lucene.queries.function.TestFunctionScoreQueryBooleanTopHit.testTopHitWithSkipIndexedValueField(TestFunctionScoreQueryBooleanTopHit.java:67)

Why it looks unintended

  • score() never reads the wrapped score, yet the bound multiplies by it: 0.0 for documents scoring 1 to 100, against the Scorer#getMaxScore contract.
  • #16431's own test expects maxScore >= 5000f for this shape, but its field has no skip index, so the check never ran; made live, it fails.
  • IndexSearcher promises that scoreDocs "is always accurate".
The full argument292 words
  • Contracts. Scorer#getMaxScore: "Return the maximum score that documents between the last target that this iterator was shallow-advanced to included and upTo included" (Scorer.java:75-79). Scorable#setMinCompetitiveScore: the scorer "may safely ignore all documents whose score is less than the given minScore" (Scorable.java:45-54); doc 4999 scores 100 and was ignored at 4.687542. IndexSearcher: "The TopDocs#scoreDocs array is always accurate however" (IndexSearcher.java:59-64). DoubleValues#getMaxScore: "the maximum score that documents ... can produce" (DoubleValues.java:55-61), i.e. the value bound already is the score bound.
  • The test framework enforces it. AssertingScorer asserts score <= getMaxScore(docID()) (AssertingScorer.java:130), CheckHits.checkTopScores asserts score <= maxScore (CheckHits.java:766), and AssertingScorer asserts that scores are needed when advanceShallow or getMaxScore is called (:100, :116; LUCENE-8314), so asking the unscored wrapped scorer for bounds is itself off-contract.
  • Same-PR tests. testMaxScoreDelegationWithDocValuesSkipper (>= 5000f), testMaxScoreMonotonicDecreasingFunction (>= 1000f) and testCustomHomemadeIncreasingAndDecreasingFunction (>= 22.3f) all wrap a TermQuery and expect a bound of at least the largest value, but only when a skip index exists, and none of their fields has one (TestFunctionScoreQuery.java:385-420, :484-487, :583-586). With the field switched to indexedField (plus forceMerge(1) and upTo = maxDoc - 1), all three fail on main, e.g. "Expected maxScore >= 1000f but got 0.0", and pass with the fix.
  • Review history. The #16431 review (19 comments, 14 inline comments, 15 reviews) discussed value monotonicity, naming and constant functions; nobody mentioned the wrapped query's factor or its unscored weight. Before #16431 the bound was always +Infinity, which is correct; 10.5.2 still returns the right hit.
  • Legal trigger. NumericDocValuesField.indexedField, the field sources, FunctionScoreQuery and BooleanQuery are public and not experimental, and nothing documents a limitation.
The devil's-advocate check169 words

The strongest case for "intended": the product is deliberate. A comment in testConstantLambdaMonotonicity says the bound is "(5.0 * innerMaxScore)" (TestFunctionScoreQuery.java:705-707), and the unscored wrapped weight is deliberate too: it saves work and testScoreMode requires it (:303-323).

Why it fails: both choices can be deliberate and still not combine. score() never multiplies by the wrapped score, so with an unscored TermQuery the comment's own formula gives 5.0 * 0 = 0, below the 5.0 the test's documents score. Three assertions in the same PR expect a bound of at least the largest value with a TermQuery; they simply never ran. No javadoc, CHANGES or MIGRATE.md text restricts the query to a top-level position, to MatchAllDocsQuery, or to fields without a skip index, and the default search(query, 1) returns the wrong hit. Verdict: a defect (the cross-model review, the intended-behaviour check and the adjudication). Scope, not intent: a top-level function query returns the right hits, and the MUST + MUST case depends on window alignment.

Suggested fix

Recommended A: do not consult the unscored wrapped scorer

When the value source does not need scores, stop consulting the unscored wrapped scorer: advanceShallow uses only the value source and getMaxScore uses a factor of 1 (+9/-2, FunctionScoreQuery.java). The repro, the finder's and reviewers' tests and the queries module pass (514 tests, 0 failures; six logged runs, assertions on and off); the trace still prunes.

final boolean innerScores = valueSource.needsScores();
// advanceShallow
int scoreShallow = scores.advanceShallow(target);
if (innerScores == false) {
  return scoreShallow;
}
return Math.min(in.advanceShallow(target), scoreShallow);
// getMaxScore, after the +Infinity check
float innerMaxScore = innerScores ? in.getMaxScore(upTo) : 1f;
return innerMaxScore * valueMaxScore * boost;
All options and trade-offs3 options · 644 words

Three variants were run; all fix the repro. They differ in what they do with the unscored wrapped scorer and with score-reading sources. A regression test for the PR is in the first option's notes. A over B is clear; A over C is a judgement about custom score-reading sources.

A: do not consult the unscored wrapped scorer (validated by the checks) (recommended)

innerScores = valueSource.needsScores(); when false, advanceShallow returns the value source's boundary and getMaxScore multiplies by 1 instead of the wrapped scorer's max score. Score-reading sources (boostByValue, boostByQuery, DoubleValuesSource.SCORES) keep today's behaviour. Regression test: the test above as a new class, and six skip-index tests of #16431 in TestFunctionScoreQuery switched to a skip-indexed, single-segment field (NumericDocValuesField.indexedField, forceMerge(1); getMaxScore(maxDoc - 1) in five of them), which makes its three dead assertions live.

final boolean innerScores = valueSource.needsScores();
// advanceShallow
int scoreShallow = scores.advanceShallow(target);
if (innerScores == false) {
  return scoreShallow;
}
return Math.min(in.advanceShallow(target), scoreShallow);
// getMaxScore, after the +Infinity check
float innerMaxScore = innerScores ? in.getMaxScore(upTo) : 1f;
return innerMaxScore * valueMaxScore * boost;

Trade-offs

  • The decision reads the rewritten source's needsScores(), while createWeight used the original source's; they agree for every source in Lucene, but passing the inner score mode into the weight would make the link explicit.
  • The live #16431 assertions need forceMerge(1): they read only the first leaf, and with a random seed (513883939B3B8CD) newIndexWriterConfig() made several segments, so one failed even with the fix. upTo = maxDoc lies past the last document and always yields +Infinity since #16697, hence maxDoc - 1.

What the checks validated

  • Standalone harness, patched class on the stock jars: this repro 2/2 (assertions on and off), the finder's repro 3/3, the independent reproduction's tests 6/6 (on and off), the GPT-5.6 reviewers' tests 4/4 each, the intended-behaviour check's class 5/5 including its AssertingQuery probe.
  • gradle, scratch worktree of ea57360912ed with the fix's test changes: without the fix 5 tests fail (this repro's 2, the 3 live #16431 assertions); with it :lucene:queries:test reports 514 tests, 0 failures (5-6 skipped) for seeds 513883939B3B8CD and DEADBEEF (assertions on and off) and two random seeds: six logged runs (a seventh is recorded without its log; the fact check re-ran two more); :lucene:queries:check -x test passes (google-java-format, ecjLint, forbiddenApis).
  • The diagram probe with the finder's variant (same bounds here): bounds 1.0 and 100.0; search(query, 1) returns doc 4999 and still counts only "4097+" hits.

B: factor 1 in getMaxScore only (the finder's fix) (validated by the checks)

float innerMaxScore = valueSource.needsScores() ? in.getMaxScore(upTo) : 1f; and nothing else.

Trade-offs

advanceShallow still calls the unscored wrapped scorer, which AssertingScorer.advanceShallow rejects (assert canScore): the intended-behaviour check's AssertingQuery probe fails 1 of 5 with it. Only visible with assertions on and when the wrapped query is itself an AssertingQuery (newSearcher does not do that: the test above under newSearcher passes 10 of 10 with this option), but a reviewer will likely ask for the first option.

What the checks validated

The finder's repro 3/3, the independent reproducer's tests through an equivalent one-line variant of the change (its root-cause shadow), the GPT-5.6 reviewers' tests, upstream TestFunctionScoreQuery 17/17 and TestDoubleValuesSource 11/11; the registry re-run 3/3.

C: drop the wrapped factor altogether (validated by the checks)

advanceShallow returns scores.advanceShallow(target) and getMaxScore returns valueMaxScore * boost, following score() and the DoubleValues#getMaxScore javadoc literally.

Trade-offs

For a score-reading source with a finite bound it trusts the source's own bound instead of multiplying. No source in Lucene reports a finite bound while needing scores (only FieldValuesSource computes a finite bound in core, and it needs no scores; DoubleValues.withDefault only delegates), so the change is invisible today but is a semantic decision for custom sources.

What the checks validated

This repro 2/2, the intended-behaviour check's class 5/5, :lucene:queries:test 514/514 for two seeds (assertions on and off).

How it was found

An AI tool formed the hypothesis while modelling FunctionScoreQuery's bound: the wrapped factor in getMaxScore has no counterpart in score(), and the wrapped weight is unscored. An SMT check confirmed the arithmetic, a randomized top-k-versus-exhaustive differential found 3,946 mismatching query configurations, and a public-API JUnit reproduced it.

The method in full207 words

Method: semi-formal (a hunt unit, function queries and block-max pruning, 2026-10-26).

  • Model. Writing the model's file:line table for FunctionScoreQuery's scorer showed that getMaxScore multiplies by in.getMaxScore(upTo) while score() never reads the wrapped score, and that createWeight weights the wrapped query without scores for field sources.
  • SMT. z3 4.12.2 and cvc5 1.4.2 (floating-point theory; versions as declared by the hunt and printed by the installed solvers) found the bound unsound when the wrapped max score is 0 (query Q1a: sat). The 0 was an input taken from reading TermQuery, so this step checked the arithmetic, not where the 0 comes from.
  • Differential. A randomized harness compared TOP_SCORES with exhaustive top-k for k = 1, 10, 50 and checked score <= getMaxScore over random indexes: 3,946 mismatching top-k query configurations on stock main (450 complete indexes), 554 with this fix alone (450); the remaining mismatches have other causes; one job each of the first two levels was cut by the job timeout; none without a skip index or for boostByValue/boostByQuery.
  • Repro. A directed public-API JUnit then failed on the stock main jars with assertions on and off. A cross-model refute review (OpenAI GPT-5.6) reproduced it with its own tests before the candidate was written.

Prior reports, history and checks

Prior reports

No prior report found (2026-10-26): 165 logged queries over Lucene issues, PRs, commits, mailing lists, JIRA, Elasticsearch, OpenSearch, Solr and the web, plus a full read of every apache/lucene issue, PR and comment since the code was written; 23 hits read, none is this bug. No open PR touches the code.

Searches5 searches

The code exists only since 2026-08-22, so the full read of GitHub activity since then covers every place a GitHub report could be. The only on-topic items are the feature request #16429, the introducing PR #16431 and the follow-up #16697; none mentions the wrapped query's factor.

What was searched, when, and what turned up
WhereWhenHowFound
GitHub issues, PRs and commits (apache/lucene); Elasticsearch, OpenSearch, Solr2026-10-26a scripted prior-report search in three passes (class and method names; symptoms and trigger API such as indexedField, fromLongField, "getMaxScore returns 0"; the composition angle such as MaxScoreBulkScorer, setMinCompetitiveScore, non-scoring wrapped query) plus manual searchesno report; on-topic only #16429, #16431, #16697
GitHub, full read through the API2026-10-26all 570 apache/lucene issues and PRs updated since 2026-08-15, all 1,038 issue comments and 689 review comments since 2026-08-20, the changed files of all 167 open PRs updated since 2026-08-22no report; no open PR touches FunctionScoreQuery.java, DoubleValuesSource.java, DoubleValues.java or their tests
PR conversations2026-10-26#16429 (body, 3 comments), #16431 (body, 19 comments, 14 inline comments, 15 reviews), #16697 (body, 2 comments)review covered monotonicity, naming and constant functions; nothing on the wrapped factor
Mailing lists (dev@, java-user@, issues@, commits@)2026-10-26FunctionScoreQuery, getMaxScore, DoubleValuesSource, boostByValue, block-max over 2026-07-15 to 2026-10-27only notification mirrors of the on-topic items and unrelated items
JIRA archive and the web2026-10-26the tool's JIRA queries; 21 web queriescontext only: LUCENE-9393 (#10433), #12297/#12383 (the 0 dummy scorer), LUCENE-8314 (#9361)

History

Introduced on main by #16431 (0d4f93a9ff9, 2026-08-22), which replaced the +Infinity bound with the product; the wrong top hit is visible end to end since #16697 (d150ba27d11, 2026-10-09), which fixed an exception in front of it. No release is affected; still present on main a9779ef3c193 (2026-10-26).

Versions tested6 rows

Introduced by apache/lucene#16431 "Enable Block-Max WAND dynamic pruning for FunctionScoreQuery via DocValuesSkipper (+422% QPS)" (0d4f93a9ff9, by @rajat315315 with @uschindler as co-author, related issue #16429, merged 2026-08-22). The unscored wrapped weight is older (LUCENE-7623, 2017, which created the class; its ScoreMode form from LUCENE-4100, 2017; the current lines from LUCENE-9393, 2020), and so is the 0 max score of an unscored TermQuery (#12383, 2023); neither is a bug on its own. From the introducing commit until #16697 (d150ba27d11, by @easyice, 2026-10-09) the tested SHOULD query threw inside the skipper first (a separate advanceShallow defect that #16697 fixed), although the bound was already 0; other query shapes in that window were not run. CHANGES.txt lists both under Lucene 11.0.0; branch_10x, branch_10_6 and branch_10_5 do not contain #16431. The commit-level runs are plain javac builds on the class path, not gradle builds; at d150ba27d11 they match the gradle-built main jars message for message.

Versions and branches executed, with results
Version or commitVerdictResult
main ea57360912ed, a9779ef3c193 affectedwrong top hit and bound 0.0 (ea57360912ed: assertions on and off; a9779ef3c193: the finder's test, assertions on)
d150ba27d11 (#16697) affectedsame messages as main
0d4f93a9ff9 (#16431) introducing commitbound 0.0; the tested query threw in the skipper (AssertionError with -ea, EOFException with -da); with #16697's change shadowed on top, the silent wrong hit appears, and the fix makes it pass
a1fcd50799f (parent of #16431) bisectbound +Infinity, right top hit
10.0.0, 10.5.2 not affectedbound +Infinity, right top hit
9.12.3 not applicableno NumericDocValuesField.indexedField (compile error)

Independent checks

An independent reproducer, a devil's advocate and OpenAI GPT-5.6 reviewers each reproduced it on main with their own tests and found the "intended" case weak; an adjudication re-ran the independent reproduction's tests and the finder's repro and confirmed it with high confidence. This doc's fix validation found that the live #16431 assertions need forceMerge(1).

Checks and verdicts383 words

Every check ran its own code on the stock main jars; all controls (no skip index, MatchAllDocsQuery as the wrapped query, the function query alone) passed.

Each independent check and its verdict
CheckVerdictWhat it found
Finder (a hunt unit) candidateSMT check and differential; the directed JUnit fails 3/3 on main with assertions on and off; its fix passes 3/3.
Refute review (OpenAI GPT-5.6, executed) real (high)Own tests on 137 and 8,203 documents: "bound 0.0 must cover actual score 146.0", and a MUST conjunction whose top hit is doc 0 instead of doc 8202; controls and 10.5.2 pass; the fix passes.
Independent reproducer realOwn tests on 10,000 documents, default codec pinned: 4 of 6 fail with assertions on and off (bound, CheckHits, SHOULD, MUST + MUST); 20 of 30 over 5 random-seed iterations; 10.5.2 6/6 pass.
Intended-behaviour review (OpenAI GPT-5.6, executed) real (high); intended weakOwn bound and default-API tests fail on main (search(query, 1) returns doc 0 instead of doc 4999); re-run separately with the same results.
Devil's advocate realNo intended, documented or misuse argument survives; three #16431 assertions made live fail on main and pass with the fix.
Adjudication confirmed (high)Re-ran the independent reproduction's tests with a fresh seed and assertions off (4 of 6 fail) and the finder's repro (3/3 fail); corrected the MUST + MUST wording.
Follow-up and registry re-run new; security noneHistory sweep (25 runs), prior-report search, security assessment; fresh re-runs fail 3/3 on main and pass with the fix; also reproduces on main a9779ef3c193.
This doc's fix validation fix validated; test correctedThe recommended fix passes the queries module (514 tests, 0 failures) in six logged runs, plus a seventh recorded without its log; the live #16431 assertions failed once with the fix under a multi-segment random seed until forceMerge(1) was added.
Fact checks and diagram check text corrected; bug confirmedA Claude fact check re-ran the repro through gradle and the runner and found two text errors (body:b's bound after doc 4096 is 0, not 3.6875415; the reason given for avoiding newSearcher was wrong). An OpenAI GPT-5.6 fact check found no error that changes the conclusion. The diagram check passed. All corrections are on this page.

Diagram file

The diagram as a still image, generated with AI and checked against the code. It is under 10 MB, the limit for a GitHub attachment.

Sources and provenance

Finding record, Lucene and upstream sources, repro files and diagram notes1,455 words

Finding record

LFV-211, snapshot of 2026-11-06

Registry title
FunctionScoreQuery's block-max bound multiplies by the wrapped query's no-score max score (0 for a TermQuery), so a BooleanQuery that contains it can drop the true top hits
Area
queries / FunctionScoreQuery block-max pruning (#16431)
Priority
Medium, rank 102: On main since #16431 FunctionScoreQuery's block-max bound multiplies by the wrapped TermQuery's no-score max of 0, so a BooleanQuery with a skip-indexed function clause silently loses its true top hits with ordinary values, the new feature's main use case, to fix before 11.0.
Impact class
silent wrong results
Impact note
Applications on unreleased main (11.0.0-SNAPSHOT, since #16431) that build FunctionScoreQuery(q, fromLongField, fromIntField, fromFloatField, fromDoubleField or fromField with INCREASING/DECREASING) around a wrapped query whose no-score max score is below 1 (TermQuery reports 0; a ConstantScoreQuery or BooleanQuery of such clauses too), on a numeric field indexed with the opt-in doc-values skip index (NumericDocValuesField.indexedField), and use it as a scoring clause of a BooleanQuery: the default IndexSearcher.search(q, n) silently returns wrong top hits (5,000 docs: doc 0 with score 4.69 instead of doc 4999 with score 100), with the default codec and searcher, -ea or -da, at a few thousand documents and with ordinary non-negative values; a top-level FunctionScoreQuery alone still returns the right hits (only its exposed getMaxScore is 0), MatchAllDocsQuery as the wrapped query and the boostByValue/boostByQuery helpers are not affected, and no released version is affected.
Summary
You index a popularity field with a skip index (NumericDocValuesField.indexedField("popularity", v)) and search BooleanQuery(SHOULD FunctionScoreQuery(TermQuery(body:a), DoubleValuesSource.fromLongField("popularity")), SHOULD TermQuery(body:b)) for the top hit, expecting the same hit as an exhaustive search, because the function query scores each matching document by its popularity value. Instead, with 5,000 documents that all match body:a, where only doc 4999 has popularity 100 (all others 1) and only doc 0 matches body:b, TOP_SCORES and the plain IndexSearcher.search(q, 1) return doc 0 (score 4.69) instead of doc 4999 (score 100). The cause is that the scorer's getMaxScore returns innerMaxScore * valueMaxScore * boost, where innerMaxScore is the max score of the wrapped TermQuery, which FunctionScoreQuery itself weighted without scores (so it reports 0), although score() is value * boost and never reads the wrapped score; every block therefore reports a bound of 0, and MaxScoreBulkScorer stops scoring documents that only the function clause makes competitive.
Introduced
0d4f93a9ff93ccc396c1323f26416b57d8dc08d9 on main (apache/lucene#16431 'Enable Block-Max WAND dynamic pruning for FunctionScoreQuery via DocValuesSkipper (+422% QPS)', by rajat315315 with uschindler as co-author, related issue #16429, merged 2026-08-22) [exec: the follow-up history sweep shows the 0.0 bound at 0d4f93a9ff9, d150ba27d11 and main with -ea and -da, while the parent a1fcd50799f, 10.0.0 and 10.5.2 report +Infinity and return the right top hit; read: git log -S innerMaxScore on FunctionScoreQuery.java returns only 0d4f93a9ff9, and git blame of FunctionScoreQuery.java:254-268 is that commit]; it replaced the scorer's +Infinity bound with innerMaxScore * valueMaxScore * boost, where the inner scorer comes from the COMPLETE_NO_SCORES weight that createWeight has used for score-independent sources since LUCENE-4100 (4fc5a872ded, 2017; the current lines 111-117 are from LUCENE-9393, e1a97a0f1ea, 2020), and a TermQuery weighted that way has reported a max score of 0 since #12383 (40ee6e583e1, 2023, a dummy no-score SimScorer); the silent wrong top hit became reachable end to end with #16697 (d150ba27d11, by easyice, 2026-10-09), which fixed a separate advanceShallow defect that made the tested SHOULD query throw first [exec: #16697's DoubleValuesSource change shadowed onto 0d4f93a9ff9 gives main's wrong top hit, and adding the fix makes all 3 tests pass] (2026-08-22 (0d4f93a9ff9 on main); silent wrong top hit reachable end to end from 2026-10-09 (d150ba27d11, #16697) for the executed SHOULD query shape)
First affected version
unreleased (main only; 11.0.0 would be the first release with it) [exec: 10.0.0 and 10.5.2 correct; read: no release tag contains 0d4f93a9ff9, and branch_10x, branch_10_6 and branch_10_5 do not contain #16431]
Affected versions
main (11.0.0-SNAPSHOT) from 0d4f93a9ff9 (2026-08-22) through 645d4fe14a2 (upstream main head, re-run 2026-10-29) [exec: Scorer#getMaxScore 0.0 at 0d4f93a9ff9, d150ba27d11 and main, -ea and -da; silently wrong TOP_SCORES top hit at d150ba27d11 and main, and at 0d4f93a9ff9 only with #16697's change shadowed, because before #16697 the tested SHOULD query threw in the skipper first (AssertionError with -ea, EOFException with -da)]; not affected: 10.0.0 [exec, -ea] and 10.5.2 [exec, -ea and -da] (FunctionScoreQuery's bound is +Infinity there) and the parent commit a1fcd50799f [exec]; not applicable: 9.12.3 [exec: compile error, no NumericDocValuesField.indexedField; the bound is +Infinity, read]; unreleased branch_10x, branch_10_6 and branch_10_5 do not contain #16431 [read; exec 2026-10-29 on the branch_10_6 RC1 4ae946e5a9b snapshot jars: not affected, -ea and -da]
Status
documented (brief, checks and a per-finding doc exist)
Verification
Confirmed by AI-run checks: an independent repro, a cross-model review that agreed, an intended-behaviour check, a prior-report search and a skeptical audit; not yet triaged by a person.
Found by
semi-formal: code reading; also reached by Z3/cvc5, fuzzing/PBT
Upstream and prior reports
none found (2026-10-26, the prior-report record: 165 logged read-only queries, 121 through a scripted prior-report search over apache/lucene issues and PRs, commits, the dev@, java-user@, issues@ and commits@ lists, the LUCENE JIRA archive, the Elasticsearch, OpenSearch and Solr repositories and the web, and 44 manual ones; plus a full read through the GitHub core API of all 570 apache/lucene issues and PRs updated since 2026-08-15, all 1,038 issue comments and 689 review comments since 2026-08-20, and the changed-file lists of all 167 open PRs updated since 2026-08-22; 23 hits read, none of them this bug; the finder's scripted prior-report search and manual searches and the independent reproduction's and the intended-behaviour check's searches found nothing either); not fixed on upstream main (HEAD ea57360912ed on 2026-10-26: FunctionScoreQuery.java is byte-identical to the tested worktree and getMaxScore still returns innerMaxScore * valueMaxScore * boost; no open PR touches FunctionScoreQuery.java, DoubleValuesSource.java, DoubleValues.java or their tests); origin: #16431 (related issue #16429; its review discussed value-function monotonicity, naming and constant functions, and nobody mentioned the innerMaxScore factor or the COMPLETE_NO_SCORES inner weight); #16697 (easyice) fixed other defects of the same feature in DoubleValuesSource and does not touch FunctionScoreQuery; context, not reports: #10433 (LUCENE-9393, the inner weight's score mode), #12297 and #12383 (TermQuery's no-score dummy scorer that returns 0), #9361 (LUCENE-8314, AssertingScorer checks that scores are needed when advanceShallow and getMaxScore are called)
Lucene commit tested
main ea57360912ed (2026-10-24; tested 2026-10-26), seed DEADBEEF and random seeds, assertions on and off, through gradle and through a standalone JUnit harness; main a9779ef3c193 (2026-10-26, same FunctionScoreQuery.java); 10.5.2, 10.0.0 and 9.12.3 jars; the introducing commit 0d4f93a9ff9, its parent and d150ba27d11 (commit-level builds); the fix in a scratch worktree of ea57360912ed.

Lucene and upstream sources

Finding record
  • a Lucene main checkout lucene/queries/src/java/org/apache/lucene/queries/function/FunctionScoreQuery.java:84-85 (boostByValue uses MultiplicativeBoostValuesSource, needsScores true at :310), :111-117 (createWeight weights the wrapped query with COMPLETE_NO_SCORES unless both the caller and the source need scores), :243-252 (score() returns (float) (value * boost), never the wrapped score), :254-259 (advanceShallow), :262-268 (getMaxScore returns innerMaxScore * valueMaxScore * boost)
  • a Lucene main checkout lucene/core/src/java/org/apache/lucene/search/TermQuery.java:75-92 (no-score TermWeight: dummy SimScorer returning 0f, 'not expected to be called since scores are not needed'); lucene/core/src/java/org/apache/lucene/search/DoubleValuesSource.java (FieldValuesSource.getMaxScore returns the decoded skipper bound; needsScores false)
  • a Lucene main checkout lucene/core/src/java/org/apache/lucene/search/Scorer.java:75-79 (getMaxScore contract), Scorable.java:44-53 (setMinCompetitiveScore: may ignore documents scoring below minScore; no-op by default, which FilterScorer inherits), MaxScoreBulkScorer.java:498-504 (clauses become non-essential while the summed bounds stay below minCompetitiveScore), BlockMaxConjunctionBulkScorer.java:160-166 (a window whose bound is below minCompetitiveScore is skipped), IndexSearcher.java:100 (TOTAL_HITS_THRESHOLD = 1000)
  • a Lucene main checkout lucene/queries/src/test/org/apache/lucene/queries/function/TestFunctionScoreQuery.java:385-417 (testMaxScoreDelegationWithDocValuesSkipper: 'assertTrue(maxScore >= 5000f)' only inside 'if (getDocValuesSkipper("val") != null)', on a field without a skip index), :635 and :707 (testConstantLambdaMonotonicity comment '(5.0 * innerMaxScore)'); lucene/test-framework/src/java/org/apache/lucene/tests/search/CheckHits.java:766 and AssertingScorer.java:100 and :130 (canScore and score <= maxScore oracles); lucene/CHANGES.txt:223 (Lucene 11.0.0: GITHUB#16431, GITHUB#16697)
The finding in 30 seconds
  • Lucene main ea57360912ed: lucene/queries/src/java/org/apache/lucene/queries/function/FunctionScoreQuery.java:109-117 (createWeight), :241 (FilterScorer), :243-252 (score), :262-269 (getMaxScore)
  • Lucene main ea57360912ed: lucene/core/src/java/org/apache/lucene/search/TermQuery.java:77-91; DoubleValuesSource.java:572-584 and :603-606; ScoreMode.java:25-26
  • Lucene main ea57360912ed: lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java:100, :598-599; TopScoreDocCollector.java:154-170; TopScoreDocCollector.java:58-59; MaxScoreBulkScorer.java:28, :36-39, :441-442, :487-493, :494-510; Scorable.java:45-54
Who is affected
  • Lucene main ea57360912ed: lucene/queries/src/java/org/apache/lucene/queries/function/FunctionScoreQuery.java:84-86, :309-312 (MultiplicativeBoostValuesSource needs scores)
  • Lucene main ea57360912ed: lucene/CHANGES.txt:223-224 (Lucene 11.0.0)
Why it looks unintended
  • Lucene main ea57360912ed: lucene/core/src/java/org/apache/lucene/search/Scorer.java:75-79; Scorable.java:45-54; IndexSearcher.java:59-64; DoubleValues.java:55-61
  • Lucene main ea57360912ed: lucene/test-framework/src/java/org/apache/lucene/tests/search/AssertingScorer.java:100, :116, :130; CheckHits.java:766
  • Lucene main ea57360912ed: lucene/queries/src/test/org/apache/lucene/queries/function/TestFunctionScoreQuery.java:303-323, :385-420, :484-487, :583-586, :705-707
Suggested fix
  • Lucene main ea57360912ed: lucene/core/src/java/org/apache/lucene/search/DoubleValuesSource.java:556-569 (getLevel), :572-584
History
  • Lucene main ea57360912ed: lucene/CHANGES.txt:223-224

Other statements trace to the hunt's working files (models, logs, reviews and briefs), which are not published.

Repro files

About the diagrams

A picture of the 5,000 documents along a doc-id axis with the two clauses, the skip index's ranges, the collector's threshold after 1,000 hits and the windows MaxScoreBulkScorer scores or skips; every number is taken from the instrumented run of the repro index. The PNG is the final summary card.

Built 2026-11-06 from the hunt's record of this finding and its doc file. Lucene main under test in this hunt: ea57360912ed. Every listed finding was re-run on Lucene main e1faeba08bde on 2026-11-02.