CombinedFieldQuery reports a NaN max score under AxiomaticF2LOG/F2EXP, so top-k search skips the best hits

CombinedFieldQuery's scorer reports simScorer.score(Float.POSITIVE_INFINITY, 1) as the highest score it can produce (CombinedFieldQuery.java:416). SimScorer#score requires a finite, positive freq and documents score(Float.MAX_VALUE, 1) as the bound. For the core similarities AxiomaticF2LOG and AxiomaticF2EXP, which the class javadoc counts as compatible, the infinite freq gives Infinity / Infinity = NaN, so getMaxScore returns NaN. When such a query is one SHOULD clause next to another scoring clause, MaxScoreBulkScorer reads the NaN as "this clause alone can never reach the minimum competitive score", makes the other clause required, and skips every document that matches only the CombinedFieldQuery. On 3,000 documents IndexSearcher#search(q, 10) returns doc 0 (2.1975672) first while the exhaustive search ranks doc 2000 (6.383921) first, and Lucene's own oracle CheckHits.checkTopScores fails. Returned scores are correct and nothing throws with assertions off. Introduced by #13999 (10.1.0, then in the sandbox module); every release from 10.1.0 to 10.6.0 (10.6.0 by reading) and main is affected; the fix is to pass Float.MAX_VALUE, as MaxScoreCache does. No upstream report was found.

• CombinedFieldQuery (BM25F): scores a term over several fields as if they were one combined field, with the searcher's similarity. @lucene.experimental, in core since 10.2.0 (a sandbox class in 10.1.0).
• Max score: Scorer#getMaxScore(upTo) is "the maximum score that documents between the last target ... and upTo" can get (Scorer.java:75-79). Top-k scorers use it to skip documents.
• Minimum competitive score: once IndexSearcher#search(q, n) has counted more than 1,000 hits (IndexSearcher.java:100), the collector tells the scorer to skip documents scoring below Math.nextUp of the n-th best score so far (TopScoreDocCollector.java:154-164).
• Essential and required clauses (MAXSCORE): clauses whose summed max scores stay below the minimum are non-essential; a competitive document must match an essential clause. With a single essential clause, if the others cannot add enough, the best non-essential clause becomes required.

1. CombinedFieldScorer sets this.maxScore = simScorer.getSimScorer().score(Float.POSITIVE_INFINITY, 1L); (CombinedFieldQuery.java:416) and returns it from getMaxScore for every upTo (:446-448). Documents are scored with their real, finite freq, so returned scores are right.
2. AxiomaticF2LOG and AxiomaticF2EXP compute tfln = freq / (freq + s + s * docLen / avgFieldLength) (AxiomaticF2LOG.java:62, AxiomaticF2EXP.java:72): Infinity / Infinity = NaN, and Math.max(0, NaN) stays NaN (Axiomatic.java:107-116).
3. A TOP_SCORES disjunction of 2 or more SHOULD clauses with minimumShouldMatch <= 1 runs on MaxScoreBulkScorer (BooleanScorerSupplier.java:308-328), and so does one with FILTER clauses and minimumShouldMatch exactly 1 (:233, :345, :364). With FILTER clauses and minimumShouldMatch 0 it runs on ReqOptSumScorer (:176); the security probe found that shape wrong too (mechanism not traced).
4. partitionScorers makes a clause non-essential only while (float) sumUpperBound(sum, n) < minCompetitiveScore (MaxScoreBulkScorer.java:500-502). A sum with NaN is never less, so the CombinedFieldQuery stays essential; in the window of doc 2000 the TermQuery's max score, 0.00022233339 (from the block impacts), is below the minimum, so there it becomes non-essential. Its bound over the whole segment, 0.0003335001, is above the minimum, and from doc 2818 on it is essential again.
5. With a single essential clause the loop tests (float) maxPossibleScoreWithoutPreviousClause >= scorable.minCompetitiveScore (:540). For NaN that is false, so the TermQuery becomes required and, in those windows, documents that match only the CombinedFieldQuery are skipped.

Doc 0 is returned only because it is the first document, collected before pruning starts. AxiomaticF2EXP behaves the same: doc 0 (3.8859417) instead of doc 2000 (11.288641); fixed bound 12.932503.

A lone CombinedFieldQuery (it uses its own bulk scorer, CombinedFieldQuery.java:397); two or more CombinedFieldQuery clauses, one per term as in the search package docs (all NaN and all essential, so no CombinedFieldQuery hit is lost; a third clause with a finite bound can still be pruned); a CombinedFieldQuery as a FILTER or MUST_NOT clause (scored with COMPLETE_NO_SCORES, BooleanWeight.java:57-59); a DisjunctionMaxQuery parent (it collected every match: pruning off). All executed.

• Who is exposed. Applications that build a CombinedFieldQuery in code (no stock Lucene query parser creates one) and score with AxiomaticF2LOG or AxiomaticF2EXP, on 10.1.0 (sandbox class), 10.2.0 to 10.5.2, or main. The open PR #14262 would make QueryBuilder build CombinedFieldQuery instances without touching the max-score line, so if merged more applications could reach this (not tested).
• Which queries. In every executed wrong case exactly one CombinedFieldQuery was among the scoring clauses of a pruned query, next to another scoring clause. That is necessary, not sufficient: hits are lost only where the other clauses' window max score falls below the minimum competitive score, and a DisjunctionMaxQuery parent or a minimumShouldMatch=2 disjunction returned correct hits with -da. Executed wrong: one CombinedFieldQuery as a SHOULD clause next to a TermQuery or a FeatureField query, also with FILTER or MUST_NOT clauses or nested in a MUST conjunction (the security probe: 7 of its 11 shapes gave a wrong top hit with a total-hits threshold of 1). Executed correct: a lone CombinedFieldQuery, one per term (2 or more clauses), a FILTER CombinedFieldQuery, a plain conjunction, a DisjunctionMaxQuery, a minimumShouldMatch=2 disjunction (-da).
• Default or opt-in. Opt-in on both sides: the default BM25 gives a finite bound, and CombinedFieldQuery is @lucene.experimental. Nothing else is special: default codec, plain IndexSearcher, IndexSearcher#search(q, n).
• Scale. No large index is needed: 3,000 documents with the default API, which must count more than 1,000 hits before pruning starts (the lost document comes after that); 301 with a total-hits threshold of 1. Single- and multi-segment indexes fail the same way under AxiomaticF2LOG (AxiomaticF2EXP ran on one segment only).
• What you see. The wrong best hits, correct scores for what is returned, no exception with assertions off. With assertions on, a minimumShouldMatch=2 disjunction trips assert Float.isNaN(maxScore) == false in WANDScorer (WANDScorer.java:91); with -da that shape only loses pruning.
• Downstream (code reading only): Elasticsearch's combined_fields query rejects every similarity but BM25; OpenSearch's has no similarity check, but no Axiomatic similarity was found in stock OpenSearch; Solr does not use the query.
• How likely. Plausible (registry likelihood): the pairing is rare, but once chosen the failure is deterministic with ordinary data. Registry priority: Medium.

# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
    => java.lang.AssertionError: top hit with pruning expected:<256> but was:<0>
       org.junit.Assert.fail(Assert.java:89)
       org.junit.Assert.failNotEquals(Assert.java:835)
       org.junit.Assert.assertEquals(Assert.java:647)
       org.apache.lucene.search.TestCombinedFieldQueryNaNMaxScore.testPrunedTopHit(TestCombinedFieldQueryNaNMaxScore.java:67)

The contract, in its own words (at ea57360912ed):

• SimScorer#score (Similarity.java:201-221): "freq is the document-term sloppy frequency and must be finite and positive"; "the maximum score that this scorer can produce is bound by score(Float.MAX_VALUE, 1)". MaxScoreCache.java:48, the sandbox TermAutomatonScorer.java:364 and BaseSimilarityTestCase.java:382 all use Float.MAX_VALUE.
• Scorer#getMaxScore (Scorer.java:75-79) returns "the maximum score" of the documents in range; NaN is not a maximum of finite scores.
• CombinedFieldQuery javadoc (CombinedFieldQuery.java:60-64): compatibility requires additive norms, which "hold for all similarities that don't customize Similarity#computeNorm"; Axiomatic extends SimilarityBase and keeps it. The query "allows other similarities besides BM25Similarity" (:72-75).
• CHANGES for #13999 (lucene/CHANGES.txt:2087-2088): "CombinedFieldQuery now returns non-infinite maximum scores, making it eligible to dynamic pruning."

Tests. AssertingSimilarity asserts Float.isFinite(freq) (AssertingSimilarity.java:76-77); a probe that wraps the default BM25 in it fails at CombinedFieldQuery.java:416 on every scored search. BaseSimilarityTestCase asserts that score(Float.MAX_VALUE, 1) is not NaN and bounds the score for every core similarity, F2LOG and F2EXP included (BaseSimilarityTestCase.java:382-391), and RandomSimilarity rotates both into random tests (RandomSimilarity.java:126-127).

Why the tests miss it. #13999 added no test, and TestCombinedFieldQuery.randomCompatibleSimilarity() picks only BM25, Boolean, Classic, LMDirichlet and LMJelinekMercer (TestCombinedFieldQuery.java:392-401); for those score(+Infinity, 1) is finite or +Infinity, never NaN.

History. #13999's description: "returning the maximum score that the sim scorer can return, which in the case of BM25 is bounded by the IDF". Its discussion covers a benchmark and a slowdown; nothing about non-finite freqs or other similarities.

The strongest case for "works as designed": both CombinedFieldQuery and the Axiomatic similarities are @lucene.experimental; the upstream test only covers BM25, Boolean, Classic and the LM similarities; and in 2017, in LUCENE-8083 (#9131), a committer wrote that AxiomaticF2EXP and AxiomaticF2LOG "can't work with maxScore" and should be removed or moved out (#9058).

Why it fails:

1. @lucene.experimental allows API changes, not wrong top hits.
2. The upstream test list is test scope, not a documented exclusion; the javadoc's only rule includes Axiomatic.
3. The 2017 remark was superseded: #9058 (LUCENE-8010) was resolved as fixed in 2018 and the similarities stayed in core, #9131 was closed as invalidated by the indexing of impacts, and today BaseSimilarityTestCase asserts a non-NaN score(Float.MAX_VALUE, 1) bound for them, which TermQuery already prunes with.
4. The caller breaks the documented precondition, not the similarity.

Conceded: for the 21 probed configurations where score(+Infinity, 1) is +Infinity (Classic, LM, DFI, AxiomaticF1/F3, IB with the log-logistic distribution) the bound is conservative and results stay correct; only pruning is lost, and no pruning ratio is promised. That is a performance remark, not this bug; the bug is the NaN, which is not an upper bound and loses hits.

The devil's advocate, the OpenAI GPT-5.6 reviewers and the adjudication found the case weak.

One option was validated with gradle in a scratch worktree at ea57360912ed (removed after the runs); the others were not run.

One line in each copy of CombinedFieldScorer's constructor (core :416, sandbox :459). SimScorer#score promises that this value bounds every score, BaseSimilarityTestCase checks the promise for every core similarity, and MaxScoreCache relies on it for TermQuery.

// CombinedFieldQuery.CombinedFieldScorer (core :416 and sandbox :459)
// was: this.maxScore = simScorer.getSimScorer().score(Float.POSITIVE_INFINITY, 1L);
this.maxScore = simScorer.getSimScorer().score(Float.MAX_VALUE, 1L);

Trade-offs

• No bound change for BM25 and the 40 DFR configurations: their score(+Infinity, 1) and score(Float.MAX_VALUE, 1) are equal in the finder's probe of 75 configurations, which uses one synthetic set of statistics (an observation, not a proof).
• For the 21 configurations whose score(+Infinity, 1) is +Infinity (Classic, the 3 LM similarities, the 3 DFI ones, AxiomaticF1/F3, the 10 IB ones with the log-logistic distribution) the bound becomes finite, so pruning can apply to them too; for Classic it is still huge (6.08e19), so little changes there. No benchmark was run.
• Not covered: CombinedFieldScorer.freq() multiplies term frequencies by field weights in float (:426-428), and weights are only checked to be at least 1, so a weight near Float.MAX_VALUE can make a document's freq +Infinity; the freq < 0 overflow check (:429) cannot catch that. A separate edge, untested.

What the checks validated

• Gradle, stock: the test fails 5 of 6 with assertions on and off.
• Gradle with the fix: the test passes with assertions on and off; TestCombinedFieldQuery, TestMaxScoreBulkScorer, TestBooleanQuery, the 7 TestAxiomatic* classes and the new test: 11 classes, 80 tests, 0 failures (seed DEADBEEF); the new test and TestCombinedFieldQuery with assertions off (seed CAFEBABE): 20 tests, 0 failures.
• Gradle with the fix: all of org.apache.lucene.search.* in :lucene:core, 214 classes, 2,336 tests, 0 failures, 49 skipped; ./gradlew :lucene:core:check -x test passes.
• Sandbox: TestCombinedFieldQuery (14 tests) passes; the sandbox-class variant passes 5 of 5 with the fix and fails 5 of 5 with only the sandbox line reverted.
• The repro runner with only the core class shadowed by the fixed one (the sandbox hunk is covered by the gradle runs): the finder's, the independent reproducer's and the devil's advocate's tests pass, and this doc's test passes 6 of 6. In the query-shape probe all 6 AxiomaticF2LOG shapes give the correct top hits; the probe run is recorded as failing, as intended, because its own check that the stock bug is still observable fails under the fix.
• Not run: other modules, a performance benchmark, x86.

After computing the bound, fall back to Float.POSITIVE_INFINITY when it is NaN, so a custom similarity that breaks the score(Float.MAX_VALUE, 1) contract gets an unpruned but correct clause.

Trade-offs

One comparison per scorer. Defends against a contract violation that BaseSimilarityTestCase already rules out for core similarities; optional.

For example in MaxScoreBulkScorer.updateMaxWindowScores and WANDScorer.

Trade-offs

Would also cover other scorers that might return NaN, but touches the hot path of every disjunction and hides a broken bound instead of fixing it. The rounding fix proposed for the same loop in LFV-060 (MathUtil.sumUpperBound at :540) does not help here: (1 + 2b) * NaN is NaN (MathUtil.java:177-193).

The 10.0.0 behaviour; the one-hunk revert on 10.1.0 passes the finder's test.

Trade-offs

Removes the #13999 pruning for every similarity, BM25 included.

Method: code reading (the hunt's non-formal tier), a hunt unit (DisjunctionMax, Synonym and CombinedField queries), 2026-10-26.

• The finder, an AI tool, listed every max-score expression in the three classes and checked each against the SimScorer#score contract ("finite and positive" freq; bound score(Float.MAX_VALUE, 1)). CombinedFieldScorer passes Float.POSITIVE_INFINITY.
• A probe of score(+Infinity, 1), score(Float.MAX_VALUE, 1) and the maximum over finite freqs for 75 core similarity configurations gave NaN for AxiomaticF2EXP and AxiomaticF2LOG and +Infinity for 21 others (a conservative bound: correct results, only no pruning of that clause).
• A deterministic public-API JUnit test on the stock main jar then showed the NaN bound and the wrong pruned top hit (2 of 2 fail, 100 of 100 over 50 iterations).
• The candidate record says it was not found by SMT. The unit's SMT proofs and bounded-exhaustive harnesses covered the DisjunctionMaxQuery bound and SynonymQuery impact merging, not CombinedFieldQuery. Independent checks followed (below).