FunctionScoreQuery and FunctionQuery score a function value of -0.0 as -0.0, so the top-1 hit is not the head of the top-2

LUCENE-7996 (8.0.0) forbids negative scores and made FunctionScoreQuery and FunctionQuery score 0 when the function value is negative. Both clamps are ordinary comparisons (factor >= 0, val >= 0 == false; explain uses < 0), and in Java -0.0 >= 0 is true, so a function value of exactly -0.0 is returned as a score of -0.0f. Lucene's test framework rejects that score (Float.compare(score, 0f) >= 0), and since 10.2.0 the hit collector disagrees with itself about it: its competitiveness test score <= topScore treats -0.0 and +0.0 as equal while its heap orders -0.0 below +0.0. With doc 0 = -0.0 and doc 1 = +0.0, search(q, 1) returns doc 0 but search(q, 2) returns doc 1 first, and searchAfter or offset paging can skip or repeat zero-scored hits. With Sort.RELEVANCE the order is consistent, but from 8.0.0 on a -0.0 hit sorts after the +0.0 hits it should tie with. In every executed check only hits whose correct score is 0 are affected; match sets, counts and positive-scored hits are unchanged. Every release from 8.0.0 to 10.6.0 has the -0.0 score; a four-line fix makes the clamps sign-aware.

LUCENE-7996: "Queries are now required to produce positive scores." and "FunctionQuery and FunctionScoreQuery now return a score of 0 when the function produces a negative value." (lucene/CHANGES.txt:6636, :6732-6733); MIGRATE: "FunctionScoreQuery maps negative values to 0." (lucene/MIGRATE.md:1368-1375).

• FunctionScoreQuery.score (FunctionScoreQuery.java:244-251): if (factor >= 0) { return (float) (factor * boost); }, otherwise // default: missing value, negative value or NaN return 0;.
• FunctionQuery.score (FunctionQuery.java:131-136): if (val >= 0 == false) { return 0; } else { return boost * val; }.
• explain truncates only value < 0 (FunctionScoreQuery.java:191, FunctionQuery.java:146), so it reports -0.0 too.

A value of -0.0 passes each test, and -0.0 times a finite non-negative boost is -0.0. The sign survives whenever the function score reaches the collector without being summed: for example at top level, under BoostQuery, as the only clause of a BooleanQuery, next to MUST_NOT clauses only, as the MUST clause next to SHOULD clauses for documents that no SHOULD clause matches (ReqOptSumScorer.java:245 starts from the required score and adds the optional one only on a match, :254-256), or inside an outer FunctionScoreQuery that multiplies by the inner score (DoubleValuesSource.SCORES, boostByValue). Conjunction scorers and pure disjunctions sum into a double that starts at +0.0 (ConjunctionScorer.java:57-61, DisjunctionSumScorer.java:41-44), and 0.0 + -0.0 is +0.0, so a MUST + FILTER query or a SHOULD-only query loses the sign.

TopScoreDocCollector rejects a hit when score <= topScore (TopScoreDocCollector.java:113), reasoning that "a document with equal score to pqTop.score cannot compete since HitQueue favors documents with lower doc Ids" (:122-124); 0.0f <= -0.0f is true. Its heap (a TernaryLongHeap on main) stores NumericUtils.floatToSortableInt(score) (DocScoreEncoder.java:31-33), the order of Float.compare; HitQueue, the collector's heap in 10.2.x, orders with Float.compare too (HitQueue.java:76-82).

• k = 1: doc 0 (-0.0) takes the single slot, topScore becomes -0.0, and doc 1 (0.0) is rejected by 0.0 <= -0.0. Result [doc 0].
• k = 2: both documents get a slot, and the heap sorts doc 1 (0.0) above doc 0 (-0.0). Result [doc 1, doc 0].

searchAfter has the same split (score > afterScore || (score == afterScore && doc <= afterDoc), TopScoreDocCollector.java:103): with page size 2, values [-0.0, +0.0] page as [doc 1, doc 0], then [doc 1] again, and values [-0.0, +0.0, 0.5] page as [doc 2, doc 1] and never show doc 0. Paging with Sort.RELEVANCE is unaffected (RelevanceComparator uses Float.compare), but its order is not the intended one either: there a -0.0 hit sorts after every +0.0 hit in every release from 8.0.0, so on the repro's index search(q, 2, Sort.RELEVANCE) = [doc 1, doc 0] instead of [doc 0, doc 1].

Lucene's test framework asserts Float.compare(score, 0f) >= 0 (AssertingScorer.java:131), so searches through newSearcher that collect such a score throw AssertionError: -0.0 with assertions on (since 10.3.0).

• Trigger. A function value of exactly -0.0 that reaches the collector unchanged: a float or double field holding -0.0 (Float.parseFloat("-0") returns it), a negated zero, or zero times a negative factor (-1f * 0f), including boostByValue with a -0.0 factor. Default codec, IndexWriterConfig and searcher; the function query itself is an explicit choice from the queries module.
• Effect. Only among hits whose correct score is 0. With Sort.RELEVANCE (every release from 8.0.0) a -0.0 hit consistently sorts after every +0.0 hit instead of tying with it in doc-id order. With score-sorted search (since 10.2.0) the result also depends on k: search(q, k) lists depend on k, searchAfter can skip a -0.0 hit for good while repeating +0.0 hits, and offset paging can show a -0.0 hit twice while never showing another zero-scored hit.
• Scale (randomized probe, 300 small multi-segment indexes with deletions, three searcher kinds, top-level shape): 18,268 of 44,204 top-k comparisons returned a different set of hits (6,564 of them from its Sort.RELEVANCE searches) and 11,590 more a different order; every lost place was lost by a -0.0 document; 2,588 of 4,080 searchAfter runs and 1,500 of 4,080 offset-paging runs repeated or dropped a hit. The probe's value mix (3 of 11 values are -0.0) is not a real-world frequency.
• Not affected: match sets, count(), totalHits, filters, deny rules, deleted documents, and every positive-scored hit.
• Test suites that use newSearcher can get AssertionError: -0.0 with assertions on, since 10.3.0, in tests that collect such a score.
• Downstream (read, not tested): in Apache Solr, edismax's boost parameter and {!boost} wrap the query in boostByValue, and {!func} builds a FunctionQuery.
• How likely: rare. It needs an exact -0.0 that survives to the collector, and then a ranking or paging that depends on the order of zero-scored hits; the symptom is easy to miss.

Doc 0 holds FloatDocValuesField("f", -0f), doc 1 FloatDocValuesField("f", 0f); the query is FunctionScoreQuery(MatchAllDocsQuery, fromFloatField("f")). The test asserts that the head of search(q, 2) (expected) is the hit of search(q, 1) (actual); its message prints the top-2 with scores, which shows the -0.0. Deterministic, single thread; the class's randomized default codec is not expected to matter (FloatDocValuesField stores the raw float bits; the test failed with each of the three seeds run).

Style: new MatchAllDocsQuery() is deprecated in favour of MatchAllDocsQuery.INSTANCE, and three lines exceed 100 columns (./gradlew tidy rewraps them); neither changes the result.

Regression test: this same class; it passes with the recommended fix.

cp $REPRO/TestFunctionScoreNegativeZero.java $LUCENE/lucene/queries/src/test/org/apache/lucene/queries/function/
cd $LUCENE && ./gradlew :lucene:queries:test --tests TestFunctionScoreNegativeZero -Ptests.seed=DEADBEEF -Ptests.asserts=true
# assertions off fails the same way: -Ptests.asserts=false

# gradle, stock 645d4fe14a20, -Ptests.seed=DEADBEEF -Ptests.asserts=true (stack trimmed; -Ptests.asserts=false identical)
> Task :lucene:queries:test FAILED

TestFunctionScoreNegativeZero > testTopOneIsHeadOfTopTwo FAILED
    java.lang.AssertionError: top-2: [doc=1 score=0.0 shardIndex=-1, doc=0 score=-0.0 shardIndex=-1] expected:<1> but was:<0>
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:33EE9A3808BAB8DA]:0)
        at org.apache.lucene.queries.function.TestFunctionScoreNegativeZero.testTopOneIsHeadOfTopTwo(TestFunctionScoreNegativeZero.java:47)

# standalone runner, seed DEADBEEF: main 645d4fe14a20 and 10.6.0 fail with -ea and -da, 10.2.0 fails, 10.1.0 passes

# an instrumented run on the same build (not part of the repro; 'PROBE ' prefix dropped)
Q score doc 1 0.0 bits 0x00000000 Float.compare(score, 0f) 0
Q score doc 0 -0.0 bits 0x80000000 Float.compare(score, 0f) -1
Q search(q, 1) [doc 0 score -0.0] totalHits 2 hits
Q search(q, 2) [doc 1 score 0.0, doc 0 score -0.0] totalHits 2 hits
Q k=1 collect doc 1 score 0.0 bits 0x00000000 -> heap top doc 0 score -0.0 (topScore -0.0) all [doc 0 score -0.0]

• The rule is documented: lucene/CHANGES.txt:6636 and :6732-6733, lucene/MIGRATE.md:1368-1375 ("Scorers are no longer allowed to produce negative scores ... FunctionScoreQuery maps negative values to 0."). The clamp's comment: "default: missing value, negative value or NaN" (FunctionScoreQuery.java:250).
• Same commit, sign-aware: a8a63464e7d (LUCENE-7996) added the clamps, AssertingScorer's assert Float.compare(score, 0f) >= 0 (AssertingScorer.java:131) and BoostQuery's Float.compare(boost, 0f) < 0 (BoostQuery.java:40), with a test that requires new BoostQuery(q, -0f) to throw (TestBoostQuery.java:35-36).
• Precedent: #16137 (open) reports a score from IBSimilarity/DistributionSPL that trips the test framework; the issue calls it negative, and a commenter (iprithv) found that it is -0.0; its fix #16139 (merged 2026-09-10, CHANGES under 10.6.0) uses Math.max(0.0, ...), which maps -0.0 to +0.0. The PR body says a negative score "breaks the contract that Lucene scores must be non-negative"; the approving review says it "is generally going to round -0.0 to 0.0".
• The upstream test with -0.0: TestFunctionScoreQuery#testTruncateNegativeScores indexes -2 and passes; a verifier's copy with -0.0 fails at AssertingScorer.java:131 on main and 10.5.2 and passes with the fix.
• The ranking is wrong under any reading: a top-1 hit that the top-2 ranks second contradicts the collector's own comment at TopScoreDocCollector.java:122-124.

The strongest case for "intended": -0.0 is numerically zero and not < 0; the Scorable#score and FunctionScoreQuery javadocs state no sign rule; FunctionQuery is @lucene.experimental; AssertingScorable and the #16139 test assert only score >= 0, which -0.0 passes; and the 19 comments of LUCENE-7996 (#9044) never mention signed zero.

Why it fails: the author of the clamps wrote the rule sign-aware in the same commit (AssertingScorer, the BoostQuery(-0f) test), upstream treated a -0.0 score as a contract break in #16137/#16139, and the top-k and paging results are inconsistent whatever -0.0 means. Verifiers and the adjudication rejected the intended-behaviour case, and the cross-model reviews rated it weak. Expect the semantic question anyway: in #16137 romseygeek asked for discussion before clamping; here the clamp already exists and the fix only makes it sign-aware.

One fix was written and validated in a scratch worktree of 645d4fe14a20; the collector-side alternative and the PayloadScoreQuery sibling are described but not run.

The change is +4/-4 lines in FunctionScoreQuery.java and FunctionQuery.java. A function value of -0.0 now takes the existing return 0 path, and explain its existing "truncated score" branch; Double.compare/Float.compare order NaN above 0, so NaN keeps its own branches. Every other value is unchanged.

// FunctionScoreQuery.score()
if (factor > 0) { // also maps -0.0 to +0.0
// FunctionScoreQuery.explain()
if (Double.compare(value, 0) < 0) { // negative values and -0.0 (NaN compares greater)
// FunctionQuery.score()
if (val > 0 == false) { // covers NaN, negative values and -0.0 (returned as +0.0)
// FunctionQuery.explain()
if (Float.compare(expl.getValue().floatValue(), 0f) < 0) { // negative values and -0.0

Trade-offs

• Same number of comparisons per document; no benchmark was run.
• Covers FunctionScoreQuery with any DoubleValuesSource (including boostByValue) and FunctionQuery; not other -0.0 producers (a similarity, the PayloadScoreQuery sibling, custom scorers).

What the checks validated

Gradle in a scratch worktree of 645d4fe14a20, OpenJDK 25.0.3, seed DEADBEEF (2026-10-31). Stock: the minimal test fails with assertions on and off, the finder's 3-test class fails 3 of 3. With the fix:

• the minimal test passes with assertions on and off;
• the finder's 3-test class (it also covers FunctionQuery) passes 3 of 3;
• a verifier's upstream-style class (newSearcher plus QueryUtils.check, which goes through AssertingScorer and compares explain() with the score) passes 6 of 6;
• the whole :lucene:queries:test passes: 51 classes, 512 tests, 0 failures, 5 skipped;
• checkGoogleJavaFormat, ecjLintMain and forbiddenApisMain for queries pass.

Earlier: 74 upstream function-query tests, the adjudication's run of a verifier's 6 tests, the same change shadowed into 8.0.0 (6/6) and the security probe (0 differences) all pass with it.

Make TopScoreDocCollector's competitiveness test (:113) and searchAfter test (:103) use the heap's order (Float.compare or the encoded long), and TopDocs.merge's score comparison (TopDocs.java:114-120) likewise. Suggested by the adjudication, a verifier and the security screen.

Trade-offs

• Covers every -0.0 producer for top-k and paging consistency.
• Does not remove the -0.0 score, the -0.0 explanation or the test-framework failure.
• Touches the collector's hot path, and the minimum-competitive-score step cannot tell the zeros apart (Math.nextUp(-0f) equals Math.nextUp(0f)), so with pruning active a +0.0 hit could still be skipped.

PayloadScoreQuery.getPayloadScore clamps with if (score >= 0 == false) (PayloadScoreQuery.java:260, added for LUCENE-7996). A custom float PayloadDecoder returning -0.0 gives a -0.0 score there (one executed check; not verified as a separate bug). score > 0 == false would match the recommended fix.

Method: non-formal (code reading guided by a differential test, confirmed by execution).

• Signal. The unit's bounded-exhaustive TopDocs.merge differential had a signed-zero probe: merge and the collectors order -0.0f and 0.0f differently. Since -0.0 is outside the score contract, that is not a bug by itself.
• Question. Can a legal query produce -0.0? Reading the LUCENE-7996 clamps found factor >= 0 and val >= 0 == false.
• Confirmation. A public-API JUnit (two documents) failed 3/3 on main with assertions on and off; then an independent reproducer, a devil's-advocate verifier, cross-model reviews (OpenAI GPT-5.6) and an adjudication checked it, and a follow-up ran the version history, a prior-report search and a security probe.