DisjunctionMaxQuery and the LatLon distance feature query accept NaN and produce NaN scores

DisjunctionMaxQuery throws IllegalArgumentException("tieBreakerMultiplier must be in [0, 1]") for -0.01, 1.01 and +/-Infinity, and LatLonPoint.newDistanceFeatureQuery throws "pivotDistance must be > 0" for 0 and -1. Both checks are written as "reject if out of range" (x < 0 || x > 1, x <= 0), and every comparison with NaN is false, so NaN passes both; +Infinity <= 0 is false too, so the pivot check also accepts +Infinity.

The query is then built and its hits score NaN: the dis-max score is max + others * tie, NaN even for a document that matches one clause (in segments where at least two clauses have a scorer), and the distance score weight * pivot / (pivot + distance) is NaN for a NaN or +Infinity pivot, although its javadoc promises a score in [0, weight]. A NaN query is not equal to itself, because equals compares the value with ==, so a query cache cannot reuse an entry for an equal copy of a NaN dis-max filter, and top-k pruning stops raising its threshold. The tie-breaker is reachable from XML query parser text (tieBreaker="NaN"). In the study's probe on main (11 query shapes) the match set never changed and nothing was written; 8.x releases returned no top-k hits for the NaN dis-max query. It is in every release since 8.0.0 and on main. Adding Float.isNaN(x) || to the two tie-breaker checks and Double.isFinite(pivot) == false || to the pivot check fixes it; it passes the repro, the closest suites and check.

1. The checks. if (tieBreakerMultiplier < 0 || tieBreakerMultiplier > 1) throw new IllegalArgumentException("tieBreakerMultiplier must be in [0, 1]"); (search/DisjunctionMaxQuery.java:65-67; the same check in the package-private DisjunctionMaxScorer constructor, search/DisjunctionMaxScorer.java:49-51). if (pivotDistance <= 0) throw new IllegalArgumentException("pivotDistance must be > 0, got " + pivotDistance); (document/LatLonPointDistanceFeatureQuery.java:58-60), right after GeoUtils.checkLatitude/checkLongitude on the origin (:54-55), which do reject NaN. For NaN every comparison is false, so the value passes; +Infinity <= 0 is false too, while +Infinity > 1 is true, so only the pivot check accepts +Infinity.
2. The dis-max score. With at least two clauses that have a scorer in a segment, the weight builds a DisjunctionMaxScorer (search/DisjunctionMaxQuery.java:158); with one, it returns that clause's scorer unchanged (:145-146), so scores stay finite in such segments. DisjunctionMaxScorer.score returns (float) (scoreMax + otherScoreSum * tieBreakerMultiplier) (search/DisjunctionMaxScorer.java:60-73). For a document matching one clause otherScoreSum is 0, and 0 * NaN is NaN. explain uses the same formula (search/DisjunctionMaxQuery.java:239).
3. The distance score. (float) (boost * (pivotDistance / (pivotDistance + distance))) (document/LatLonPointDistanceFeatureQuery.java:280; :130 in explain): NaN for a NaN pivot, and Infinity / Infinity = NaN for a +Infinity pivot. The factory's weight is not a second gap: when it is not 1 it goes through BoostQuery (document/LatLonPoint.java:373-374), which rejects non-finite and negative boosts (search/BoostQuery.java:40).
4. equals is not reflexive. tieBreakerMultiplier == other.tieBreakerMultiplier (search/DisjunctionMaxQuery.java:353) and pivotDistance == other.pivotDistance (document/LatLonPointDistanceFeatureQuery.java:80) are false for NaN, while hashCode uses the bit pattern (:365, :89): two identical NaN queries have the same hash code and are never equal. A +Infinity pivot query stays equal to itself.
5. Query text path (dis-max only). The XML query parser reads tieBreaker with DOMUtils.getAttribute(e, "tieBreaker", 0.0f), which is Float.parseFloat (lucene/queryparser/.../xml/builders/DisjunctionMaxQueryBuilder.java:46, xml/DOMUtils.java:124-127), and passes it to new DisjunctionMaxQuery (:59). tieBreaker="NaN" parses to (f:a | f:b)~NaN, not equal to itself, while boost="NaN" and tieBreaker="2" in the same element are rejected. The pivot has no parser path.
6. A third, unchecked site. BlendedTermQuery.DisjunctionMaxRewrite(float) stores the tie-breaker with no check, compares it with == (search/BlendedTermQuery.java:165-172, 180) and later builds new DisjunctionMaxQuery with it; DisjunctionMaxRewrite(NaN) is not equal to itself.

• Who is hit. An application that lets a NaN reach the tie-breaker (a computed value such as 0f/0f, a configuration or request value "NaN" read with Float.parseFloat, or XML query text tieBreaker="NaN"), or a NaN or +Infinity reach the pivot (for example a pivot computed by dividing by zero). Default IndexWriterConfig, codec and searcher; DisjunctionMaxQuery is a mainstream core query.
• What goes wrong. No exception, then: the hits of that query score NaN (for the dis-max query, in segments where at least two clauses have a scorer), so its ranking is meaningless; top-k pruning mostly stops, because once the top-k heap holds NaN Math.nextUp(NaN) > minCompetitiveScore is false and the threshold is never raised (TopScoreDocCollector.java:160-161): in the study's probe the distance query alone counted all 200,000 matches instead of 1,169, and across the 11 query shapes the slowest NaN shape took about 8x the control's time (a dis-max of the distance query and a term, 41 vs 5 ms) while some ran faster, on a loaded host; a NaN dis-max query is not equal to any other instance, and IndexSearcher consults the query cache only when scores are not needed (IndexSearcher.java:979), so where it runs as a filter the cache cannot reuse an entry for an equal copy (the same instance still hits through the identity check, so only misses, never a wrong hit, LRUQueryCache.java:1341); the distance query is never cached (isCacheable returns false, LatLonPointDistanceFeatureQuery.java:113-114); with assertions on, a NaN dis-max inside a minimumShouldMatch disjunction trips assert Float.isNaN(maxScore) == false (WANDScorer.java:91).
• Limits. In the study's probe on main (200,000 documents in 4 segments, 11 query shapes, assertions on and off) the set of matching documents was always the same as a finite control's, no hit escaped a FILTER clause, nothing hung and nothing was written. That covers the tested shapes, not every composition. Segments where only one dis-max clause has a scorer keep finite scores.
• Older releases were worse (NaN tie-breaker, 2 matching documents): 8.0.0 and 8.11.4 return 0 top-k hits (on 8.11.4 count() is 2); 9.12.3 with assertions off returns sentinel ScoreDocs (doc 2147483647, score -Infinity) in place of one or both hits. Their mechanism was not traced, and TopScoreDocCollector says it "will not properly collect hits" with NaN scores (TopScoreDocCollector.java:30-31), so how hits are collected once NaN is accepted is not a stable contract.
• Likelihood: rare. Normal code passes constant tie-breakers such as 0.1 and fixed pivots; the recorded search found no prior report since 8.0.0 (2019).

The empty clause list is legal; the check runs before the clauses are used. A single method would stop at the first failure and never show the second site, so each check has its own one-line method.

As the regression test it needs only the ASF license header; with it, :lucene:core:check -x test passes, it fails on stock and passes with the recommended fix. The NaN scores and the +Infinity pivot are shown by the score probe, not by more tests; the recommended fix rejects +Infinity too (probe).

TestQueryRangeChecksAdmitNaN > testDistanceFeatureQueryNaNPivot FAILED
    junit.framework.AssertionFailedError: Expected exception IllegalArgumentException but no exception was thrown
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:DE978C4DE6240292]:0)
        at app//org.apache.lucene.tests.util.LuceneTestCaseParent.expectThrows(LuceneTestCaseParent.java:1432)
        at app//org.apache.lucene.tests.util.LuceneTestCaseParent.expectThrows(LuceneTestCaseParent.java:1418)
        at app//org.apache.lucene.search.TestQueryRangeChecksAdmitNaN.testDistanceFeatureQueryNaNPivot(TestQueryRangeChecksAdmitNaN.java:16)

TestQueryRangeChecksAdmitNaN > testDisjunctionMaxQueryNaNTieBreaker FAILED
    junit.framework.AssertionFailedError: Expected exception IllegalArgumentException but no exception was thrown
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:BA0373407EB7AA57]:0)
        at app//org.apache.lucene.tests.util.LuceneTestCaseParent.expectThrows(LuceneTestCaseParent.java:1432)
        at app//org.apache.lucene.tests.util.LuceneTestCaseParent.expectThrows(LuceneTestCaseParent.java:1418)
        at app//org.apache.lucene.search.TestQueryRangeChecksAdmitNaN.testDisjunctionMaxQueryNaNTieBreaker(TestQueryRangeChecksAdmitNaN.java:11)

:lucene:core:test (FAILURE): 2 tests, 2  failures

# the same two failures with -Ptests.asserts=false, and on the 10.6.0 release jars (assertions on and off)

# score probe on the same main, assertions off (identical with assertions on and on 10.6.0):
DMQ tie=0.1 accepted q=(title:x | body:x)~0.1 equalsSelf=true totalHits=2 hits hits: doc0=0.22671446 doc1=0.21363801 explain(doc1)=0.21363801
DMQ tie=NaN accepted q=(title:x | body:x)~NaN equalsSelf=false totalHits=2 hits hits: doc0=NaN doc1=NaN explain(doc1)=NaN
DMQ control tie=-0.01 rejected: tieBreakerMultiplier must be in [0, 1]
DMQ control tie=1.01 rejected: tieBreakerMultiplier must be in [0, 1]
DMQ control tie=Infinity rejected: tieBreakerMultiplier must be in [0, 1]
LatLon pivot=1000.0 accepted equalsSelf=true totalHits=3 hits hits: doc0=1.0 doc1=0.0063191317 doc2=6.3713634E-4
LatLon pivot=NaN accepted equalsSelf=false totalHits=3 hits hits: doc0=NaN doc1=NaN doc2=NaN
LatLon pivot=Infinity accepted equalsSelf=true totalHits=3 hits hits: doc0=NaN doc1=NaN doc2=NaN
LatLon control pivot=0.0 rejected: pivotDistance must be > 0, got 0.0
LatLon control pivot=-1.0 rejected: pivotDistance must be > 0, got -1.0

• The checks enforce a stated range for every other value. -0.01, 1.01, +/-Infinity and -Float.MIN_VALUE are rejected for the tie-breaker; 0, -0.0, -1 and -Infinity for the pivot (executed controls).
• The documented score range. LatLonPoint.newDistanceFeatureQuery: "score = weight * pivotDistanceMeters / (pivotDistanceMeters + distance), ie. score is in the [0, weight] range" (LatLonPoint.java:357-368).
• NaN is not a valid score. TopScoreDocCollector: "The values Float#NaN and Float#NEGATIVE_INFINITY are not valid scores. This collector will not properly collect hits with such scores." (TopScoreDocCollector.java:30-31); the test framework's AssertingScorer asserts !Float.isNaN(score) (AssertingScorer.java:129).
• equals must work for caching. Query.equals: "This is required so that QueryCache works properly" (Query.java:96-97), and Object.equals must be reflexive. Lucene's own AssertingIndexSearcher rejects a NaN dis-max query in QueryUtils.checkEqual (executed).
• The NaN-rejecting convention. BoostQuery: Float.isFinite(boost) == false || ... (BoostQuery.java:40; LUCENE-7996, a8a63464e7d, by the same committer 20 days before the dis-max check). FeatureField: pivot <= 0 || Float.isFinite(pivot) == false with the message "pivot must be > 0" (FeatureField.java:700-701). LogOddsFusionQuery: Float.isNaN(alpha) || alpha < 0 || alpha > 1 for the same [0, 1] range (LogOddsFusionQuery.java:82). GeoUtils.checkLatitude, called in the same LatLon constructor: Double.isNaN(latitude) || ... (geo/GeoUtils.java:85-86).
• Never designed. The dis-max check came as a side change of LUCENE-8097 (maxScore on disjunctions); the pivot check was copied from the long recency feature query (LUCENE-8340), where NaN cannot occur. Neither issue mentions NaN, and no upstream test passes NaN to either constructor.

The strongest case: no constructor documents @throws, the DisjunctionMaxQuery javadoc states no range at all, and the caller passed NaN, so garbage in, garbage out. The cross-model review rated this argument moderate.

Why it fails: the checks clearly mean to enforce the range their messages state, since they reject every other out-of-range value; the codebase rejects NaN in the sibling checks and tests it; and NaN breaks two stated contracts (the [0, weight] score range, equals reflexivity). It is a good reason for low priority, not for keeping NaN.

The recommended fix was run in a scratch worktree of 645d4fe14a20 (OpenJDK 25.0.3, seed DEADBEEF), removed afterwards. "Closest suites" means TestDisjunctionMaxQuery, TestLatLonPointDistanceFeatureQuery, TestBlendedTermQuery, TestSimpleExplanations, TestComplexExplanations, TestSimpleSearchEquivalence, TestLRUQueryCache and TestBooleanQuery in lucene/core, plus the XML query parser's TestCoreParser.

One line in each of DisjunctionMaxQuery, DisjunctionMaxScorer and LatLonPointDistanceFeatureQuery, messages unchanged. The same shape as LogOddsFusionQuery, GeoUtils.checkLatitude (in the same LatLon constructor) and FeatureField; > 1 already rejects +Infinity for the tie-breaker.

--- a/lucene/core/src/java/org/apache/lucene/search/DisjunctionMaxQuery.java
-    if (tieBreakerMultiplier < 0 || tieBreakerMultiplier > 1) {
+    if (Float.isNaN(tieBreakerMultiplier) || tieBreakerMultiplier < 0 || tieBreakerMultiplier > 1) {
--- a/lucene/core/src/java/org/apache/lucene/search/DisjunctionMaxScorer.java
-    if (tieBreakerMultiplier < 0 || tieBreakerMultiplier > 1) {
+    if (Float.isNaN(tieBreakerMultiplier) || tieBreakerMultiplier < 0 || tieBreakerMultiplier > 1) {
--- a/lucene/core/src/java/org/apache/lucene/document/LatLonPointDistanceFeatureQuery.java
-    if (pivotDistance <= 0) {
+    if (Double.isFinite(pivotDistance) == false || pivotDistance <= 0) {

Trade-offs

• A caller that passes NaN (or a +Infinity pivot) now gets an exception instead of NaN scores; no valid value changes. Constructor-time cost only.
• "pivotDistance must be > 0, got Infinity" reads oddly; "must be finite and > 0" would be clearer (cosmetic, not run).
• BlendedTermQuery.DisjunctionMaxRewrite(float) has no check: with this fix a NaN rewrite method fails later, when rewrite calls new DisjunctionMaxQuery (BlendedTermQuery.java:170-172). The same check in its constructor (:165-167) would fail where the value is set (not run).

What the checks validated

• Stock 645d4fe14a20: the repro fails (2 of 2) with assertions on and off.
• With the fix: the repro passes with assertions on and off; the 8 closest core suites pass (203 tests, 3 skipped, 0 failures); the XML query parser's TestCoreParser passes (22 tests); :lucene:core:check -x test (google-java-format, ecj lint, forbidden APIs, license checks) passes with the license-headed regression test in place, which passes with the fix and fails on stock. These are targeted runs, not the full Lucene test suite.
• Probe with the three fixed files shadowed: NaN tie-breaker, NaN pivot and +Infinity pivot are rejected; tie-breaker 0.1 and pivot 1,000 m give the same scores as stock.

if ((tieBreakerMultiplier >= 0 && tieBreakerMultiplier <= 1) == false) and if ((pivotDistance > 0) == false). It rejects NaN: with the two files shadowed on 645d4fe14a20 the minimal test passes 2 of 2 with assertions on and off, and the dis-max and distance-feature suites passed with it on the study's main.

Trade-offs

+Infinity > 0 is true, so the pivot form still accepts +Infinity and needs pivotDistance < Double.POSITIVE_INFINITY as well; the recommended form states the intent more plainly.

Makes a NaN query equal to itself.

Trade-offs

Not enough: the hits still score NaN and pruning still stops.

Rejects tieBreaker="NaN" in query text.

Trade-offs

Not enough: the programmatic path and the pivot stay open.

Method: semi-formal (recorded so because the Z3 queries confirmed a hypothesis written down beforehand).

• The unit's bytecode scan (JDK 25 ClassFile API) of all 142 concrete Query classes flagged float and double fields that equals compares with primitive == while hashCode uses the bit pattern, among them DisjunctionMaxQuery.tieBreakerMultiplier and LatLonPointDistanceFeatureQuery.pivotDistance.
• Z3 obligations over IEEE binary32/binary64 (12, all as expected; two of them concern a BayesianScoreQuery check that is not part of this finding): NaN is the only value that passes each range check while outside its stated range, the negated in-range form of the dis-max check admits no such value, and NaN is a witness that equals by == is not reflexive, while equality by bit pattern is.
• The unit's bounded-exhaustive equality harness on the stock jar reported NONREFLEXIVE for both classes; the finder reduced it to a public-API JUnit, and an independent reproducer wrote its own and found the NaN scores, the +Infinity pivot and the older-release symptoms.