A +Infinity minimum competitive score makes ScorerUtil.minRequiredScore return NaN, so top-k search can drop documents scoring +Infinity

Lucene's top-k conjunctions (BlockMaxConjunctionBulkScorer) and disjunctions (MaxScoreBulkScorer) prune candidates with ScorerUtil.filterCompetitiveHits, which keeps a document only if its partial score is at least minRequiredScore(maxRemainingScore, minCompetitiveScore, n). When the minimum competitive score is +Infinity, that helper computes +Inf - R = +Inf, subtracts Math.ulp(+Inf) = +Inf and returns NaN; nothing is >= NaN, so every candidate is dropped, including documents that already score +Infinity.

TopScoreDocCollector asks for +Infinity itself: Math.nextUp(Float.MAX_VALUE) when the k-th best score is Float.MAX_VALUE, and the raw +Infinity from another slice in a concurrent search. So a plain single-threaded search(q, 1) returns doc 0 at Float.MAX_VALUE instead of doc 1001 at +Infinity, and a concurrent search can return the wrong documents among hits tied at +Infinity. No error is raised. It is a regression from a 10.3.0 speed-up (#14827 and a same-day follow-up); 10.2.2 is correct. The trigger is rare: scores must overflow to +Infinity, for example from very large FeatureField values or boosts near Float.MAX_VALUE. Starting the threshold from Float.MAX_VALUE for a +Infinity minimum fixes it.

1. The collector asks for +Infinity. TopScoreDocCollector breaks ties by ascending doc id, so once it is pruning, a later document must score strictly more than the k-th best: it calls setMinCompetitiveScore(Math.nextUp(topScore)) (TopScoreDocCollector.java:154-164, nextUp at :160). For a k-th score of Float.MAX_VALUE that is +Infinity. In a concurrent search, a leaf with lower doc ids gets the raw score from another slice, without nextUp, so that its tied documents stay competitive (:137-152, :145); after k hits at +Infinity in that slice, that is +Infinity too.
2. The bulk scorers turn it into a cut-off. Before adding each further clause, BlockMaxConjunctionBulkScorer.scoreWindowScoreFirst calls ScorerUtil.filterCompetitiveHits(buffer, sumOfOtherClauses[i], minCompetitiveScore, n) (BlockMaxConjunctionBulkScorer.java:182-191); MaxScoreBulkScorer.scoreNonEssentialClauses does the same (MaxScoreBulkScorer.java:452-458). These are the only callers.
3. The cut-off is NaN. minRequiredScore (ScorerUtil.java:131-142) starts from minCompetitiveScore - maxRemainingScore = +Inf - R = +Inf (:133) and steps by Math.ulp(minCompetitiveScore) = +Inf (:135). The loop test minRequiredScore > 0 && (float) sumUpperBound(minRequiredScore + R, n) >= minCompetitiveScore is true (:136-138), so it computes +Inf - +Inf = NaN (:139); NaN > 0 is false, and NaN is returned (:141).
4. NaN keeps nothing. filterCompetitiveHits returns early only if (minRequiredScore <= 0) (:158), which is false for NaN, and then VectorUtil.filterByScore keeps a document only if score >= NaN (:162-163; VectorUtil.java:601-620), which is never true. The batch is emptied and none of it is collected (BlockMaxConjunctionBulkScorer.java:194-197).

The collector collects doc by doc until it has counted more than 1,000 hits (IndexSearcher.java:100, TopScoreDocCollector.java:113-120). At doc 1000 the minimum competitive score becomes nextUp(3.4028235E38) = +Infinity, and from doc 1001 on the pruning path runs (BlockMaxConjunctionBulkScorer.java:92-107). With the +Infinity document at doc 1000 or lower, the test passes.

All fail with assertions on and off; all pass with the fix.

• Who can be hit. Top-k searches (ScoreMode.TOP_SCORES; IndexSearcher#search(Query, int) is enough) that run a top-level conjunction on BlockMaxConjunctionBulkScorer or a top-level disjunction on MaxScoreBulkScorer, over documents whose scores overflow to +Infinity, when the search reaches the filter step (the conjunction scorer skips it before a clause when two consecutive remaining-score bounds are equal, BlockMaxConjunctionBulkScorer.java:184-189). The inputs pass every check: FeatureField accepts any finite positive normal value (FeatureField.java:152-172) and linear weights in (0, 64] (:593-597); BoostQuery rejects only non-finite or negative boosts (BoostQuery.java:38-43), and the classic QueryParser accepts a 38-digit boost.
• When. Single-threaded: the k-th best score is exactly Float.MAX_VALUE and a later document scores +Infinity. Concurrent: a slice with higher doc ids collects k hits at +Infinity first, so a slice with lower doc ids drops its +Infinity ties. A single-threaded search whose k-th score is already +Infinity shows nothing wrong.
• Defaults. Default IndexWriterConfig and codec, an IndexSearcher without an executor, 1,002 documents. The rare part is the input.
• What goes wrong. A silently wrong top-k, with assertions on and off: a +Infinity document loses to a Float.MAX_VALUE one, or the documented doc-id tie-break among +Infinity hits breaks. Hits below Float.MAX_VALUE rank correctly; no extra, deleted or filtered-out document ever appears (22 executed checks). With the stock collector the result is never empty: it raises the minimum to +Infinity only once a queue holds k hits whose k-th best scores Float.MAX_VALUE or more (TopScoreDocCollector.java:137-171). A custom collector that sets a +Infinity minimum before any hit collects nothing (0 of 300 in the first verifier's test).
• Plausible patterns (API reachability only; no real-world use was observed). Boosts near Float.MAX_VALUE used as a "pin this first" sentinel, or unbounded FeatureField values.

The test sits in org.apache.lucene.search, the package of the upstream TestScorerUtil, but uses only public API. Its own code draws no random values and pins the codec; it extends LuceneTestCase, whose environment is still randomized (similarity, locale, time zone), and seeds DEADBEEF and 5EED1234 and 40 random-seed iterations give the same failure, with assertions on and off. new IndexSearcher(reader) has no executor. The test does not assert a single segment; a probe that indexes the same 1,002 documents with a default IndexWriterConfig reports segments=1. All runs were on Linux aarch64 with OpenJDK 25.0.3; no other JVM or architecture was tried (the mechanism is plain IEEE 754 float arithmetic).

It exercises the conjunction path (BlockMaxConjunctionBulkScorer). The disjunction path (MaxScoreBulkScorer) shares the helper and fails the same way in the 3,001-document variant, so one test method is enough. As a regression test it is the same test: it fails on stock and passes with either fix.

TestMinRequiredScoreInfinite > testInfiniteScoreAfterMaxValueBottom FAILED
    java.lang.AssertionError: top hit 0 score 3.4028235E38 expected:<1001> but was:<0>
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:9E3F8F3100331A09]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.failNotEquals(Assert.java:835)
        at org.junit.Assert.assertEquals(Assert.java:647)
        at org.apache.lucene.search.TestMinRequiredScoreInfinite.testInfiniteScoreAfterMaxValueBottom(TestMinRequiredScoreInfinite.java:52)

• The contract. A scorer "may safely ignore all documents whose score is less than the given minScore" (Scorable.java:45-52). The minRequiredScore javadoc promises a value with (float) sumUpperBound(minRequiredScore + maxRemainingScore, n) <= minCompetitiveScore and says a non-optimal value "doesn't hurt correctness, it just means that these docs will be filtered out later" (ScorerUtil.java:123-130). NaN is no threshold, and here the documents are lost, not filtered later.
• +Infinity is not declared invalid. TopScoreDocCollector names only Float.NaN and Float.NEGATIVE_INFINITY as invalid (TopScoreDocCollector.java:30-31); +Infinity was taken off that list in LUCENE-2271 (2010). That is an omission, not an affirmative guarantee, but the collector produces +Infinity itself (next point). The test framework's AssertingScorer never checks for infinity: it asserts not-NaN, >= 0 and <= the max score for scores (AssertingScorer.java:129-131), and not-NaN and non-decreasing from 0 for minimum scores (:63, :92-93). TestWANDScorer covers +Infinity max scores (testRandomWithInfiniteMaxScore, @Nightly, :828-836).
• Lucene produces the +Infinity minimum itself, at TopScoreDocCollector.java:160 (nextUp) and :145 (the raw score that keeps lower-doc-id ties competitive).
• A regression from a speed-up. 10.2.2 and 9.12.3 pass the same test. #14827 is listed under Optimizations in CHANGES 10.3.0; its review and the two same-day follow-ups discuss finite rounding only, and no comment, commit message, test or CHANGES entry mentions infinite or NaN scores. Upstream TestScorerUtil.testMinRequiredScore draws the minimum from random().nextFloat(), in [0, 1) (TestScorerUtil.java:100-104), so it never reaches this case.

The strongest case for "intended": overflowing scores are degenerate. LUCENE-8016 (#9064, open since 2017) calls the infinite scores from boosts near Float.MAX_VALUE "trappy" and proposes bounding boosts; WANDScorer.scaleMinScore asserts Float.isFinite(minScore) (WANDScorer.java:112); FeatureField's linear function is "Expert", with "an expectation that a feature value is encoded in the index in a way that makes sense for scoring" (FeatureField.java:78-81); and the filterCompetitiveHits javadoc speaks of scores that "exceed" the minimum.

Why it fails: none of these is a documented precondition. LUCENE-8016 was never implemented, and BoostQuery and FeatureField accept the trigger inputs. The WANDScorer assert is an internal assumption that TopScoreDocCollector itself breaks (a separate bug, LFV-113). "Exceed" cannot mean that a +Infinity document must be dropped when the collector asks for exactly +Infinity, and 10.2.2 kept it. The study's reviewers rated the argument weak; it is about how rare the input is, which the low priority reflects. A later cross-model fact check of this doc (OpenAI GPT-5.6) rated it moderate: upstream could treat the fix as low-priority hardening, given the extreme inputs and no known production report, though it still judged the behaviour a real defect.

Two fixes were run through gradle in a scratch worktree of 645d4fe14a20 (OpenJDK 25.0.3, seed DEADBEEF). The recommended one is the hunt's candidate fix, with the infinity test narrowed to +Infinity.

In ScorerUtil.minRequiredScore (ScorerUtil.java:133-135). Finite minimums are unchanged; for an infinite maximum of the other clauses the start is -Infinity and nothing is filtered, as today when that maximum exceeds the minimum.

-    double minRequiredScore = minCompetitiveScore - maxRemainingScore;
+    // With minCompetitiveScore = +Infinity, starting from it would give +Inf - ulp(+Inf) = NaN.
+    // Start from the largest finite float instead; the loop below still compares against +Inf.
+    float start =
+        minCompetitiveScore == Float.POSITIVE_INFINITY ? Float.MAX_VALUE : minCompetitiveScore;
+    double minRequiredScore = start - maxRemainingScore;
     // note: we want the float ulp in order to converge faster, not the double ulp
-    double subtraction = Math.ulp(minCompetitiveScore);
+    double subtraction = Math.ulp(start);

Trade-offs

• A top-k saturated at +Infinity now keeps and scores its +Infinity candidates, as the contract requires, so it costs about as much as exhaustive scoring: 13.9-17.3 ms against 12.2-17.8 ms for COMPLETE on 400,000 documents that all score +Infinity, where the buggy path took 3.5-3.8 ms because it discarded them (rough, best of 5 on a shared host).
• Normal queries are unaffected: the new branch runs only for a +Infinity minimum.
• The candidate fix used Float.isInfinite(minCompetitiveScore), which is also true for -Infinity; then the loop would step down from Float.MAX_VALUE about 2^24 times. Lucene's collectors never pass -Infinity, but == Float.POSITIVE_INFINITY keeps today's behaviour for it. From 0 to +Infinity the two forms are identical.

What the checks validated

• Stock: the repro fails with assertions on and off.
• Fix: the repro passes with assertions on and off.
• The closest suites (TestScorerUtil, TestMaxScoreBulkScorer, TestBlockMaxConjunction, TestBlockMaxConjunctionBulkScorer, TestWANDScorer, TestBooleanQuery, TestBoolean2, TestBooleanMinShouldMatch and the repro): 94 tests, 0 failures, 4 skipped.
• :lucene:core:test --tests 'org.apache.lucene.search.*': 215 classes, 2,340 tests, 0 failures, 49 skipped.
• :lucene:core:check -x test with the repro in the tree passes (google-java-format, ecjLint main and test, forbiddenApis, licenses, checksums). The full core suite was not run.
• Earlier, the candidate's Float.isInfinite form, shadowed onto the stock jars, passed the finder's, the intended-behaviour check's and the cross-model reviewers' tests and the 19 history tests over 10.3.0, and the safety-property probe's bug detector fired in 0 of 22 checks.

In ScorerUtil.filterCompetitiveHits (ScorerUtil.java:158).

-    if (minRequiredScore <= 0) {
+    // also skip filtering when minRequiredScore is NaN (an infinite minCompetitiveScore)
+    if (!(minRequiredScore > 0)) {
       return;
     }

Trade-offs

One line, and it guards any future NaN. But minRequiredScore still returns NaN, against its own javadoc, and for a +Infinity minimum this step then prunes nothing (still correct: the collector rejects non-competitive hits). It can be combined with the recommended fix.

What the checks validated

Gradle: the repro passes, and the same closest suites pass (94 tests, 0 failures, 4 skipped). A NaN guard of the same kind also made the first verifier's 5 public-API tests pass. Not run with the whole core search package, so the recommended fix is the more strongly validated one; neither fix ran the full core suite.

Pass Float.MAX_VALUE instead of +Infinity at TopScoreDocCollector.java:145 and :160.

Trade-offs

Fixes only the stock collector: any other collector may still call setMinCompetitiveScore(+Infinity) (a custom collector in the checks collected 0 of 300 +Infinity documents), and minRequiredScore would still return NaN. Not recommended.

Method: semi-formal (a hypothesis formed while modelling, then witnessed by a tool), a hunt unit on WAND and block-max bounds.

• While modelling WAND and block-max pruning bounds (SMT lemmas for WANDScorer's scaled-integer test, which assume finite scores), an AI tool reviewed the block-max filter by code reading: a document dropped by filterCompetitiveHits must not be able to reach the minimum competitive score. That obligation, like the earlier study's proof of the two helpers, presumes a finite minimum.
• Evaluating minRequiredScore at +Infinity by hand gives +Inf - ulp(+Inf) = NaN, and filterByScore(NaN) keeps nothing. The signal came from code reading while modelling.
• A JUnit on the stock main jar witnessed it (the helper and a forced-order concurrent search); an independent verifier found the simpler single-threaded form that the minimal test is cut down from.