Late-interaction rescoring gives a document a NaN score when one of its token vectors is all zeros (default COSINE)

LateInteractionField accepts a token matrix in which one token vector is all zeros, and LateInteractionRescorer and LateInteractionFloatValuesSource use COSINE by default. The cosine of a zero vector is 0/0 = NaN, and SUM_MAX_SIM keeps each query token's best similarity with Float.max, which returns NaN as soon as one argument is NaN. So one zero row makes the document's whole score NaN, even when another of its tokens matches the query exactly: the document {{1,0},{0,0}} rescored with the query {{1,0}} scores NaN instead of 1.0 with assertions off, and throws an AssertionError in VectorUtil.cosine with assertions on (LateInteractionFloatValuesSource.java:175 at 5adff78bb168). The rescorer sorts with ScoreDoc.COMPARATOR, which orders two finite scores by score but any pair that involves the NaN by doc id alone, so the other hits can be mis-ordered too. The single-vector KnnFloatVectorField rejects a zero vector under COSINE; the late-interaction classes check only null, empty and equal dimensions. Every release since 10.3.0 has it. The API is opt-in and experimental and the input uncommon, so the priority is Low.

1. The zero row is accepted. LateInteractionField.encode checks that the matrix and its first token are non-null and non-empty and that all tokens have the same dimension, nothing else (document/LateInteractionField.java:74-102, a fix site); the javadoc asks only for that ("Value should not be null or empty. Composing token vectors for provided multi-vector value should have the same dimension.", :51-52). The query side, validateQueryVector, makes the same three checks (search/LateInteractionFloatValuesSource.java:65-80).
2. COSINE is the default of LateInteractionRescorer.create(field, query) (search/LateInteractionRescorer.java:39-41) and of new LateInteractionFloatValuesSource(field, query) (search/LateInteractionFloatValuesSource.java:45-47).
3. 0/0 = NaN. COSINE.compare is normalizeToUnitInterval(cosine(v1, v2)) (index/VectorSimilarityFunction.java:84-86). VectorUtil.cosine only does assert Float.isFinite(r) (util/VectorUtil.java:100-107, the assert at :105) and has no runtime handling of a zero norm. With assertions off it returns NaN, and Math.max((1 + NaN) / 2, 0) is NaN (util/VectorUtil.java:461-463).
4. One NaN wins. SUM_MAX_SIM (search/LateInteractionFloatValuesSource.java:154-181) does, per query token, maxSim = Float.max(maxSim, vectorSimilarityFunction.compare(q, d)) (:175, the fix site of option A) and then result += maxSim (:177). In the repro Float.max(1.0, NaN) is NaN, so the document scores NaN although its first token matched exactly.
5. The NaN becomes the hit's score and breaks the sort. DoubleValuesSourceRescorer.rescore stores it as hit.score (search/DoubleValuesSourceRescorer.java:75-76; LateInteractionRescorer.combine returns the value, search/LateInteractionRescorer.java:62-65) and sorts with ScoreDoc.COMPARATOR (search/DoubleValuesSourceRescorer.java:79-83), which tests > and < and otherwise falls back to the doc id (search/ScoreDoc.java:57-66). Both tests are false when a score is NaN, so two finite scores are ordered by score but any pair with the NaN hit by doc id alone: the order is no longer consistent. In a 3-hit probe, scores doc0 = 0.5497519, doc1 = NaN, doc2 = 0.9993762 came back in that order, the best finite hit last; with doc1's zero row dropped (or with fix A) doc1 scores 1.0 and the order is doc1, doc2, doc0. By reading only (not executed): when topN is below the number of hits, ArrayUtil.select first picks the top topN with the same comparator (search/DoubleValuesSourceRescorer.java:79-81), so a NaN hit could also change which hits survive the cut.

Related, confirmed by verifier probes but not part of the repro or of this finding: an all-zero query token makes every rescored document that has a value in the field NaN under COSINE (a hit without a value gets 0, search/LateInteractionRescorer.java:62-65); a NaN token component gives NaN under COSINE, DOT_PRODUCT and EUCLIDEAN; encode also accepts +Inf components.

• Who. Applications that rescore with the late-interaction (multi-vector, ColBERT-style) API using its default COSINE, and whose document token matrices can contain an all-zero row, for example zero padding of fixed-size token matrices. The ColBERT reference encoder returns zero rows for padded positions when padding is kept (read in its source); other toolkits were not checked.
• Default vs opt-in. The API is opt-in; LateInteractionFloatValuesSource and LateInteractionRescorer are @lucene.experimental, LateInteractionField is not tagged. Within that API, COSINE is the default.
• Effect. Silent: no exception with assertions off. The document's score is NaN, which TopScoreDocCollector documents as not a valid score (search/TopScoreDocCollector.java:30-31), and the other rescored hits can be mis-ordered.
• Not hit. DOT_PRODUCT and EUCLIDEAN give a finite score for the same zero row (1.0 in the repro). Typical normalized token embeddings have no zero rows, so accidental hits are rare.
• Scale. One document with one zero row; no concurrency, no size threshold. Every release from 10.3.0 through 10.6.0.

TestLateInteractionZeroTokenVector (one method, extends LuceneTestCase, package org.apache.lucene.search of the lucene/core tests). Line 43 is assertFalse(Float.isNaN(rescored.scoreDocs[0].score)), so the failure with assertions off means the score is NaN.

As the regression test, the same method fits in TestLateInteractionRescorer. It asserts only "not NaN", so it fits a fix that gives a finite score (options A and B); a fix that rejected the zero document token would need expectThrows instead. On 10.3.x, replace MatchAllDocsQuery.INSTANCE (added in 10.4.0) with new MatchAllDocsQuery(). An independent reproducer's test also fails with the scalar (non-SIMD) vector provider.

cp $REPRO/TestLateInteractionZeroTokenVector.java $LUCENE/lucene/core/src/test/org/apache/lucene/search/
cd $LUCENE && ./gradlew :lucene:core:test --tests org.apache.lucene.search.TestLateInteractionZeroTokenVector -Ptests.seed=DEADBEEF

# gradle, scratch worktree at 5adff78bb168, -Ptests.seed=DEADBEEF -Ptests.asserts=false
TestLateInteractionZeroTokenVector > testZeroTokenVectorDoesNotMakeScoreNaN FAILED
    java.lang.AssertionError
        at org.junit.Assert.assertFalse(Assert.java:75)
        at org.apache.lucene.search.TestLateInteractionZeroTokenVector.testZeroTokenVectorDoesNotMakeScoreNaN(TestLateInteractionZeroTokenVector.java:43)
:lucene:core:test (FAILURE): 1 test, 1  failure

# gradle, same, assertions on
TestLateInteractionZeroTokenVector > testZeroTokenVectorDoesNotMakeScoreNaN FAILED
    java.lang.AssertionError
        at org.apache.lucene.util.VectorUtil.cosine(VectorUtil.java:105)
        at org.apache.lucene.index.VectorSimilarityFunction$3.compare(VectorSimilarityFunction.java:85)
        at org.apache.lucene.search.LateInteractionFloatValuesSource$ScoreFunction$1.compare(LateInteractionFloatValuesSource.java:175)
        at org.apache.lucene.search.LateInteractionFloatValuesSource$1.doubleValue(LateInteractionFloatValuesSource.java:92)
        at org.apache.lucene.search.DoubleValuesSourceRescorer.rescore(DoubleValuesSourceRescorer.java:75)
        at org.apache.lucene.search.TestLateInteractionZeroTokenVector.testZeroTokenVectorDoesNotMakeScoreNaN(TestLateInteractionZeroTokenVector.java:42)
:lucene:core:test (FAILURE): 1 test, 1  failure

# with fix A: :lucene:core:test (SUCCESS): 1 test, assertions on and off

• Contract. "The values Float#NaN and Float#NEGATIVE_INFINITY are not valid scores." (search/TopScoreDocCollector.java:30-31). The late-interaction javadocs ask only for non-null, non-empty, equal-dimension tokens (document/LateInteractionField.java:51-52).
• Design rule and precedent. Lucene validates vectors in the field and query classes, not in the performance-sensitive similarity functions; that is why VectorUtil.cosine only asserts. #12281 (071461ece54, 2023) added those checks to KnnFloatVectorField / KnnFloatVectorQuery; in its review rmuir wrote "Checking if vectors are finite needs to be done at eg index time (in the xxField class)". Today KnnFloatVectorField rejects a zero vector under COSINE (document/KnnFloatVectorField.java:53-56, :146-151, :172-176; "zero vector not allowed with cosine similarity function" at :55 and :174, "zero vector not allowed with cosine similarity configured in field type" at :148-149) and non-finite values (:105, :151, :176); KnnFloatVectorQuery checks only that its target is finite (search/KnnFloatVectorQuery.java:97). Open issue #15540 (a zero vector under COSINE is accepted at indexing and fails only later, at merge or in CheckIndex; single-vector path) was followed by #15751 (10.5.0, the KnnFloatVectorField and KnnByteVectorField constructors), #16098 (10.5.0, setVectorValue) and #16708 (10.6.0, columnar addBatch validation), which add the zero-under-COSINE check on the single-vector indexing paths; #15751 is by the author of #14729. On #15540 mikemccand wrote "the lenient + broken behavior now is no good" and benwtrent "I agree, cosine should be validated". None of these touched the late-interaction classes.
• Tests. The upstream late-interaction tests build tokens from random().nextFloat() values, never non-finite and practically never all zero; their validation tests cover only null, empty and dimension cases (TestLateInteractionField, TestLateInteractionFloatValuesSource).
• History. #14729, which added the three classes, never discusses zero, NaN or non-finite tokens; no later commit touches the files.

The strongest case for "not a bug": the cosine of a zero vector is undefined, so NaN is garbage in, garbage out. LateInteractionField is a plain binary doc-values field that cannot know the similarity at index time, and on #15540 benwtrent wrote "I don't think we should prevent 0 vectors for similarities that support them" (DOT_PRODUCT, EUCLIDEAN), with a +1 from mikemccand. The COSINE javadoc says the preferred way is to normalize and use DOT_PRODUCT (index/VectorSimilarityFunction.java:77-80), and the API is experimental.

Why it fails: that shapes the fix (a blanket index-time rejection does not fit today's similarity-agnostic field), not the intent. Lucene picks COSINE as the default, the API accepts the input without error, NaN is documented as an invalid score, and the same input class was fixed on the single-vector indexing paths. @lucene.experimental says the API "might change in incompatible ways in the next release"; it says nothing about invalid scores. A credibility audit rated it real but low priority, in the same family as #15540; hence priority Low.

Options A and A+B were built and checked through gradle on 5adff78bb168; C was not built. A blanket index-time rejection of zero document tokens does not fit today's field: it does not know the similarity, and zero vectors are legitimate under DOT_PRODUCT and EUCLIDEAN (#15540); a redesign could carry the similarity, but the smaller fix is on the score and query side. Whether a zero token should be ignored (A) or rejected (B for the query, C for documents) is a policy choice for the maintainers. Open PR #16748 (rank SUM_MAX_SIM by raw MaxSim) edits the same line and keeps Float.max over VectorUtil.cosine, so it does not fix this; whichever lands second may need a rebase.

Under COSINE a zero token has no direction, so it cannot be a document's best match: skip it before calling compare. The repro then scores exactly 1.0 (a probe test with fix A prints score=1.0 with assertions on and off). A document whose tokens are all zero gets Float.MIN_VALUE per query token, the floor SUM_MAX_SIM already uses, also for an empty document (:160-162, :165). DOT_PRODUCT, EUCLIDEAN and MAXIMUM_INNER_PRODUCT are unchanged.

            if (vectorSimilarityFunction == VectorSimilarityFunction.COSINE
                && VectorUtil.isZeroVector(d)) {
              continue; // cosine with a zero vector is 0/0 = NaN: the token cannot match
            }
            maxSim = Float.max(maxSim, vectorSimilarityFunction.compare(q, d));

Trade-offs

isZeroVector returns at the first non-zero component, so for a typical token (first component non-zero) it costs one comparison. A is a policy choice: a zero document token silently stops counting under COSINE, and an all-zero document gets the Float.MIN_VALUE floor instead of an error. A leaves a zero query token (every document with a value NaN) and NaN components alone; option B covers those. Needs import org.apache.lucene.util.VectorUtil;.

What the checks validated

• Gradle in a scratch worktree at 5adff78bb168, seed DEADBEEF: the repro fails without the fix (assertions on and off) and passes with it (on and off).
• With the fix: TestLateInteractionField (3), TestLateInteractionFloatValuesSource (3), TestLateInteractionRescorer (2), TestRescoreTopNQuery (7, of which 2 are @Nightly and skipped) and the repro: 16 tests, 0 failures. checkGoogleJavaFormat passes.
• A probe test with fix A shadowed over the main jars prints score=1.0 for the repro's document, with assertions on and off.
• Earlier, the same logic passed the repro on the main jars with assertions on and off.

VectorUtil.checkFinite on every token in LateInteractionField.encode and in validateQueryVector, and, under COSINE, an IllegalArgumentException for an all-zero query token (the similarity is known in the constructor). This mirrors KnnFloatVectorField and KnnFloatVectorQuery. +18/-3 in two files, together with A.

Trade-offs

A behaviour change: NaN/Inf tokens are rejected at indexing and a zero query token under COSINE at construction; already-indexed non-finite tokens still score NaN. The query check runs before Objects.requireNonNull(vectorSimilarityFunction), so the existing testValidations (a null similarity with bad query vectors, expecting IllegalArgumentException) still passes. No test here covers B's new checks; the regression test stays the one repro.

What the checks validated

Same worktree with A+B: the repro passes (assertions off; on in the suite run), the same 16 tests pass and checkGoogleJavaFormat passes.

Throw IllegalArgumentException from SUM_MAX_SIM for a zero document token under COSINE.

Trade-offs

A clear signal, but one bad document fails the whole rescore, and the repro (which asserts a non-NaN score) would have to become expectThrows.

Method: semi-formal (a proof-obligation audit of a class's contract, witnessed by an executable repro).

• The finder read LateInteractionFloatValuesSource and SUM_MAX_SIM against the obligation that a rescored score is a valid score (TopScoreDocCollector: NaN is not one) and against what LateInteractionField.encode and validateQueryVector accept (null, empty and dimension checks only). VectorUtil.cosine states its precondition only as an assertion, and the single-vector entry points discharge it with explicit checks; the late-interaction entry points do not.
• The finder then wrote the one-test public-API JUnit used here, which fails on main with assertions on and off and on 10.6.0. An independent reproducer, an executed cross-model review (OpenAI GPT-5.6) and a devil's-advocate verifier reproduced it with their own tests, and an adjudication re-ran the finder's test ("Prior reports, history and checks").