PayloadScoreQuery scores a payload of -0.0 as -0.0, so the top-1 hit is not the head of the top-2

LUCENE-7996 (8.0.0) forbids negative scores, and PayloadScoreQuery enforces it itself: getPayloadScore returns 0 unless score >= 0, and explain truncates only value < 0. In Java -0.0 >= 0 is true, so a payload score of exactly -0.0 is returned and explained as -0.0. It arises when the decoder reads IEEE floats, such as PayloadHelper.decodeFloat, the inverse of FloatEncoder, and the payload function keeps a -0.0: a lone -0.0 payload under MaxPayloadFunction or MinPayloadFunction (which take the first payload as is; Min also keeps it next to +0.0 or positive payloads, Max loses it to them), or a custom PayloadFunction. The stock FLOAT_DECODER reads one byte and cannot produce it, and Sum and Average turn a lone -0.0 into +0.0. Lucene's test framework rejects that score (Float.compare(score, 0f) >= 0), and since 10.2.0 the hit collector disagrees with itself about it: its competitiveness test score <= topScore treats -0.0 and +0.0 as equal while its heap orders -0.0 below +0.0. With doc 0 = -0.0 and doc 1 = +0.0, search(q, 1) returns doc 0 but search(q, 2) returns doc 1 first. Only hits whose correct score is 0 are affected. Every release from 8.0.0 to 10.6.0 has it; a two-line fix makes the clamp sign-aware.

LUCENE-7996: "Queries are now required to produce positive scores." (lucene/CHANGES.txt:6636); MIGRATE: "Scorers are no longer allowed to produce negative scores." (lucene/MIGRATE.md:1368-1372). The follow-up commit "LUCENE-7996: PayloadScoreQuery must produce positive scores." added:

• getPayloadScore (PayloadScoreQuery.java:258-265): float score = function.docScore(...); if (score >= 0 == false) { return 0; } else { return score; }.
• getPayloadExplanation (:270): the "truncated score" branch only if (expl.getValue().floatValue() < 0), so explain reports -0.0 too.
• With includeSpanScore the score is getSpanScore() * getPayloadScore() (:291), and a positive span score times -0.0 is still -0.0.

Each payload is decoded (:226) and folded into an accumulator that starts at 0 for every document (:218-221, :227-235). MaxPayloadFunction and MinPayloadFunction return the first payload as is and Math.max/Math.min afterwards (MaxPayloadFunction.java:34-37, MinPayloadFunction.java:31-34), so a lone -0.0 survives, and Math.min keeps -0.0 next to +0.0 and positive payloads (Min over [2.0, -0.0, 3.5] gives -0.0); Math.max(-0f, 0f) is +0.0, so under Max a -0.0 survives only if no payload is +0.0 or positive (payloads [-0.0, 0.0]: Min -0.0, Max +0.0). SumPayloadFunction adds (SumPayloadFunction.java:35) and 0 + -0.0 is +0.0, so Sum and Average lose the sign of a lone -0.0; Average divides by the payload count, so in a corner case a tiny negative sum rounds to -0.0 (payloads [-1.4E-45, 0.0]). A custom PayloadFunction can return -0.0 directly.

The decoder must read the IEEE float: PayloadHelper.decodeFloat (PayloadHelper.java:60-61), the inverse of encodeFloat, which FloatEncoder uses (FloatEncoder.java:31-32). The stock PayloadDecoder.FLOAT_DECODER returns the payload's first byte (PayloadDecoder.java:29): for -0.0's bytes 80 00 00 00 that is -128, which the clamp maps to 0.

TopScoreDocCollector rejects a hit when score <= topScore (TopScoreDocCollector.java:113), reasoning that "a document with equal score to pqTop.score cannot compete since HitQueue favors documents with lower doc Ids" (:122-124); 0.0f <= -0.0f is true. Its heap stores NumericUtils.floatToSortableInt(score) (DocScoreEncoder.java:31-33), the order of Float.compare; HitQueue, the heap in 10.2.x, uses Float.compare too (HitQueue.java:76-82).

• k = 1: doc 0 (-0.0) takes the single slot, topScore becomes -0.0, and doc 1 (0.0) is rejected by 0.0 <= -0.0. Result [doc 0].
• k = 2: both documents get a slot, and the heap sorts doc 1 (0.0) above doc 0 (-0.0). Result [doc 1, doc 0]. No pruning is involved: the collector sets a minimum competitive score only after more than 1,000 hits (IndexSearcher.java:100, TopScoreDocCollector.java:155), and here there are two.

Lucene's test framework asserts Float.compare(score, 0f) >= 0 (AssertingScorer.java:131), so searches through newSearcher that collect such a score throw AssertionError: -0.0 with assertions on (since 10.3.0).

• Trigger. A payload that decodes to exactly -0.0: the token text -0.0 or -0 in a delimited float payload field, a tiny negative weight such as -1e-50 (which Float.parseFloat rounds to -0.0f), or a pipeline that writes a computed 0 * negative weight with Float.toString (-0.0). Then a payload function that keeps it (a lone -0.0 under MaxPayloadFunction or MinPayloadFunction; Min also next to +0.0 or positive payloads, Max not; a custom function; in a corner case AveragePayloadFunction), with includeSpanScore true or false, and a decoder that reads the IEEE float. No special setup: default IndexWriterConfig and a plain IndexSearcher (the tests ran with the test framework's randomized codec and directory); payload scoring itself is an explicit choice (PayloadFunction is @lucene.experimental).
• Effect. Only among hits whose correct score is 0: their score and explanation carry a minus sign, and since 10.2.0 their order depends on k (search(q, 1) is not the head of search(q, 2)). So offset paging (each page re-runs the search with a larger k and skips the earlier hits) can show the -0.0 hit on several pages while other zero-scored hits are never shown: with page size 1, payloads [-0.0], [0.0], [0.0] give doc 0 on all three pages (executed, assertions off). searchAfter paging returned each hit once in the four small shapes tried; for the FunctionScoreQuery sibling of this bug, random multi-segment indexes also showed it skipping a -0.0 hit, which was not tried here.
• Not affected: which documents match, count(), totalHits, every positively scored hit, the index. With assertions off Lucene throws nothing.
• Test suites that use newSearcher can get AssertionError: -0.0 with assertions on, since 10.3.0, in tests that collect such a score.
• Downstream (read, not run): Apache Solr's {!payload_score} parser offers func=min|max|average|sum and decodes float payload fields with PayloadHelper.decodeFloat, so the decoder this needs ships by default there, and a lone term|-0.0 value with func=max or func=min appears to reach the same -0.0 score; not reproduced in Solr.
• How likely: rare. It needs an exact -0.0 payload that survives the payload function (with several payloads, Max loses it to any +0.0 or positive payload), and then a ranking that depends on the order of zero-scored hits; the symptom is easy to miss.

Doc 0's token x carries PayloadHelper.encodeFloat(-0f), doc 1's encodeFloat(0f); the query is PayloadScoreQuery(SpanTermQuery(f:x), MaxPayloadFunction, bytes -> PayloadHelper.decodeFloat(bytes.bytes, bytes.offset), false). The test asserts that the head of search(q, 2) (expected) is the hit of search(q, 1) (actual); its message prints the top-2 with scores, which shows the -0.0. Deterministic, single thread; it failed with both seeds run (the randomized directory and codec are not expected to matter). Under newSearcher with assertions on it would throw AssertionError: -0.0 first. It checks the collector symptom, not the score itself: a collector-only change would make it pass while the score is still -0.0 (an extra assertion on the hits' score bits would pin the score too; not written or run).

The command needs core, codecs, queries, analysis-common and test-framework jars plus JUnit 4.13.2, Hamcrest 3.0 and randomizedtesting-runner 2.9.2 (from the Gradle cache).

Regression test: this same class. Inside lucene/queries, whose tests do not depend on analysis-common, replace PayloadHelper.encodeFloat(payload) with a 4-byte array filled by BitUtil.VH_BE_FLOAT.set(data, 0, payload) and the decoder with bytes -> (float) BitUtil.VH_BE_FLOAT.get(bytes.bytes, bytes.offset), which is what PayloadHelper does. That form fails through ./gradlew :lucene:queries:test on stock main and passes with the fix.

# plain javac + JUnitCore, stock 645d4fe14a20, -ea (trimmed: JDK warning, test-framework NOTE lines and stack frames removed; -da -Dtests.asserts=false identical)
JUnit version 4.13.2
.E
There was 1 failure:
1) testTopOneIsHeadOfTopTwo(org.apache.lucene.queries.payloads.TestPayloadScoreQueryNegativeZero)
java.lang.AssertionError: top-2: [doc=1 score=0.0 shardIndex=-1, doc=0 score=-0.0 shardIndex=-1] expected:<1> but was:<0>
	at __randomizedtesting.SeedInfo.seed([DEADBEEF:33EE9A3808BAB8DA]:0)
	at org.apache.lucene.queries.payloads.TestPayloadScoreQueryNegativeZero.testTopOneIsHeadOfTopTwo(TestPayloadScoreQueryNegativeZero.java:59)
FAILURES!!!
Tests run: 1,  Failures: 1

# standalone runner, seed DEADBEEF: main 645d4fe14a20 and 10.6.0 fail with -ea and -da, 10.2.0 fails, 10.1.0 passes

# an instrumented run on the same build (not part of the repro; 'PROBE ' prefix dropped)
Q doc 0 freq 1 position 0 payload bytes 80 00 00 00 decodeFloat -0.0 bits 0x80000000 FLOAT_DECODER -128.0
Q score doc 0 -0.0 bits 0x80000000 Float.compare(score, 0f) -1
Q search(q, 1) [doc 0 score -0.0] totalHits 2 hits
Q search(q, 2) [doc 1 score 0.0, doc 0 score -0.0] totalHits 2 hits

• The rule is documented: lucene/CHANGES.txt:6636, lucene/MIGRATE.md:1368-1372 ("Scorers are no longer allowed to produce negative scores"). The clamp's commit 187849f9b67 is titled "LUCENE-7996: PayloadScoreQuery must produce positive scores."
• The class breaks its own rule: a payload of -1.0 scores +0.0, -0.0 scores -0.0, and explain has a "truncated score" branch for negatives that -0.0 skips (PayloadScoreQuery.java:270-273).
• Same day, sign-aware: a8a63464e7d (LUCENE-7996, same author) added AssertingScorer's assert Float.compare(score, 0f) >= 0 (AssertingScorer.java:131) and BoostQuery's Float.compare(boost, 0f) < 0 (BoostQuery.java:40), with a test that requires new BoostQuery(q, -0f) to throw (TestBoostQuery.java:35-36).
• Precedent: #16137 (open) reports a score from IBSimilarity/DistributionSPL that trips the test framework; a commenter found that it is -0.0. Its fix #16139 (merged 2026-09-10, CHANGES under 10.6.0) uses Math.max(0.0, ...), which maps -0.0 to +0.0.
• The ranking is wrong under any reading: a top-1 hit that the top-2 ranks second contradicts the collector's own comment at TopScoreDocCollector.java:122-124.
• Not misuse: a custom PayloadDecoder is the documented extension point, and encodeFloat/decodeFloat is the documented pair. No upstream payload test uses a negative payload.

The strongest case for "intended": -0.0 is numerically zero and not < 0; the Scorable#score and PayloadScoreQuery javadocs state no sign rule; PayloadFunction is @lucene.experimental; AssertingScorable and AssertingSimilarity assert only score >= 0, which -0.0 passes; TopScoreDocCollector's javadoc names only NaN and negative infinity as invalid scores; the stock decoder cannot produce it; and the 19 comments of LUCENE-7996 (#9044) never mention signed zero.

How far it goes: for the score itself it has some weight, since MIGRATE.md says only "negative" and reading the rule as "no -0.0" is an inference. The inference is supported by the sign-aware checks the clamp's author added the same day (AssertingScorer, the BoostQuery(-0f) test), by the class clamping -1.0 but not -0.0, and by upstream treating a -0.0 score as a contract break in #16137/#16139; the trigger uses only public extension points. For the ranking it fails outright: the top-k result is inconsistent whatever -0.0 means. Verifiers and the adjudication rejected the intended-behaviour case; the cross-model review and a verifier rated it weak.

One fix was written and validated in a scratch worktree of 645d4fe14a20; the collector-side alternative is described but not run.

The change is +2/-2 lines in PayloadScoreQuery.java. A payload score of -0.0 now takes the existing return 0 path, and explain its existing "truncated score" branch; Float.compare orders NaN above 0, so NaN keeps its own branches. With includeSpanScore, spanScore * +0.0 is +0.0. Every other value is unchanged.

// PayloadScoreQuery.PayloadSpanScorer.getPayloadScore()
if (score > 0 == false) { // also maps -0.0 (and NaN, negatives) to +0
// PayloadScoreQuery.PayloadSpanScorer.getPayloadExplanation()
if (Float.compare(expl.getValue().floatValue(), 0f) < 0) { // includes -0.0

Trade-offs

• Constant time per document, as before (explain, not on the scoring path, uses Float.compare instead of <); no benchmark was run.
• Covers every PayloadFunction, built-in or custom, and every decoder; not other -0.0 producers (the FunctionScoreQuery/FunctionQuery sibling, a similarity, custom scorers).

What the checks validated

Gradle in a scratch worktree of 645d4fe14a20, OpenJDK 25.0.3, seed DEADBEEF (2026-10-31), with the repro's gradle form (the two PayloadHelper calls inlined) in lucene/queries. Stock: it fails with assertions on and off. With the fix:

• the test passes with assertions on and off;
• the payloads package passes: 7 classes, 35 tests, 0 failures, 2 skipped;
• the whole :lucene:queries:test passes: 52 classes, 513 tests, 0 failures, 5 skipped;
• checkGoogleJavaFormat, ecjLintMain, ecjLintTest, forbiddenApisMain and forbiddenApisTest for queries pass with the test in place.

The repro's own command (the unchanged class, plain javac and JUnitCore on jars built from the same worktree) fails on stock and passes with the fix, assertions on and off. Earlier: the finder's 3 tests, 32 upstream payload tests, a verifier's 7 tests and the same change shadowed into 8.0.0 (8/8) all pass with it.

Make TopScoreDocCollector's competitiveness test (:113) and searchAfter test (:103) use the heap's order (Float.compare or the encoded long).

Trade-offs

• Hides the top-k and paging symptom for every -0.0 producer.
• Leaves the -0.0 score, the -0.0 explanation and the test-framework failure, so the recommended fix is still needed.
• Touches the collector's hot path, and the minimum-competitive-score step cannot tell the zeros apart (Math.nextUp(-0f) equals Math.nextUp(0f)).

The same >= 0 idiom from LUCENE-7996 sits in FunctionScoreQuery and FunctionQuery (a sibling bug with its own four-line fix). The two fixes touch different files and can land together or separately. Open PR #15819 edits scorerSupplier in PayloadScoreQuery.java (not the clamp).

Method: non-formal first flag, then semi-formal (bounded-exhaustive translation validation, confirmed by execution).

• First flag. While verifying the FunctionScoreQuery sibling (2026-10-26), a probe with a custom float decoder returned -0.0 from PayloadScoreQuery; it was noted as an unverified sibling.
• Signal. The payload/span hunt wrote an independent reference of payload scoring (Min, Max, Sum and Average folded over the decoded payloads, the final score clamped to a non-negative float with -0.0 mapped to +0.0) and compared it, by float bits, with the stock main jar over a bounded-exhaustive space of small indexes and queries (payloads including -0.0, an IEEE-float decoder): 679,172 of 76,177,080 score checks returned -0.0 where the reference gives +0.0; all other non-nested score checks matched. So the run confirmed a suspected site at scale rather than finding it blind.
• Confirmation. A public-API JUnit failed 3/3 on main with assertions on and off; then a cross-model review with a shell (OpenAI GPT-5.6), an independent reproducer, a second such review, a devil's-advocate verifier and an adjudication checked it, and a follow-up ran the version history, a prior-report search and a security screen.