UnifiedHighlighter with maxPassages = 0 throws a NullPointerException when the field has a match to highlight

The UnifiedHighlighter keeps a field's best maxPassages passages in a java.util.PriorityQueue. When a passage is finished, FieldHighlighter.maybeAddPassage treats the queue as full when passageQueue.size() == maxPassages and then compares the passage with the lowest-scoring entry, passageQueue.peek() (FieldHighlighter.java:212 on main 645d4fe14a20). For maxPassages = 0 the still empty queue counts as full (0 == 0), peek() returns null, and .getScore() throws.

That happens for every document with a match inside the highlighted text of the field (a match past the first maxLength characters, 10,000 by default, counts as no match), through highlight, highlightFields and highlightWithoutSearcher, with offsets from postings, term vectors or re-analysis, with assertions on and off. A document without a match returns before the queue is built and gets null, so 0 works or fails depending only on whether the document matches. With per-field counts {0, 1}, when the 0 field has such a match, the whole call throws and the other field loses its snippet. Nothing checks the values. The check dates from the first UnifiedHighlighter commit (LUCENE-7438, 2016), so every release from 6.3.0 to 10.6.0 and main is affected.

1. No value check. highlight(field, query, topDocs, maxPassages) wraps the value in an array and calls highlightFields (UnifiedHighlighter.java:766-770). highlightFieldsAsObjects checks the field list and the array lengths (:892-897) but never the values, and builds one FieldHighlighter per field with its own count (:922-924). highlightWithoutSearcher does the same for one string (:1096-1107).
2. No match: no queue, null. highlightOffsetsEnums returns new Passage[0] before the queue exists when the field has no term occurrence (FieldHighlighter.java:137-139). The fallback summary then asks for maxPassages sentences when maxNoHighlightPassages is at its default -1 (:89-94), that is 0, and the result is null (:96-100). A match beyond the highlighted text counts as no match: with a searcher the stored text is cut to maxLength characters (10,000 by default, UnifiedHighlighter.java:100, :1443-1447), and an offset at or past that length ends the loop before any passage is scored (FieldHighlighter.java:164-173), so such a document also gets null (executed: a 12,005-character document whose only match starts after character 10,000).
3. Match: an empty queue that counts as full. The queue gets capacity Math.min(64, maxPassages + 1), 1 for 0, so creating it works (:141-143). When the first scored passage is finished, maybeAddPassage evaluates passageQueue.size() == maxPassages && passage.getScore() < passageQueue.peek().getScore() (:212): 0 == 0 is true, peek() of the empty queue is null, and .getScore() throws.
4. Why only 0. For every maxPassages >= 1 the queue is non-empty whenever size() == maxPassages. Negative values never get there: a capacity below 1 is rejected by the JDK constructor first (:143), with an IllegalArgumentException. Nor does Integer.MAX_VALUE: its maxPassages + 1 overflows at :143 first, a separate bug.

• What goes wrong. The call throws on the first document with a match to highlight, so the request gets no snippets; in a multi-field call every field loses its snippets. The error is an internal NullPointerException, not a clear argument error (the javadoc documents only an IllegalArgumentException for fields without offsets), so a server is likely to report it as its own failure rather than as a bad request.
• Easy to miss in tests. With 0, a document without a match returns null, so tests on non-matching data pass.
• Scale. None needed: one document, one term; deterministic.
• Downstream (code reading only, not run). Solr's unified highlighter copies the per-field hl.snippets into maxPassages without a range check, so hl.method=unified&hl.snippets=0, or f.<field>.hl.snippets=0 next to another field, would reach this code with 0 (no Solr request was run). Elasticsearch and OpenSearch map 0 fragments to Integer.MAX_VALUE - 1 and do not reach it.
• Workaround. Leave a field that should get no snippet out of the request instead of passing 0.
• How likely: rare. It needs an explicit 0; Lucene's own tests pass only 1, 2, 3 or 10 (never 0), and no report was found in ten years.

The test sits in the uhighlight test package like the upstream tests and uses a fixed StandardAnalyzer and an in-memory ByteBuffersDirectory; by code reading the failing logic does not depend on the codec, similarity or locale the test framework randomizes, and every recorded seed fails the same way. It turns the NullPointerException into an AssertionError that states what was expected (the cause keeps the FieldHighlighter frames). It passes unchanged with either fix, so it is also the regression test.

It needs UnifiedHighlighter.builder and the lucene.tests packages, so it compiles on 9.1.0 and later. Older releases (6.3.0 to 9.0.0) were checked with a separate history probe that uses the public constructor. The test's line numbers in the output include the 16-line license header.

cp $REPRO/TestZeroMaxPassages.java $LUCENE/lucene/highlighter/src/test/org/apache/lucene/search/uhighlight/
cd $LUCENE && ./gradlew :lucene:highlighter:test --tests org.apache.lucene.search.uhighlight.TestZeroMaxPassages -Ptests.seed=DEADBEEF

TestZeroMaxPassages > testZeroMaxPassagesWithMatch FAILED
    java.lang.AssertionError: maxPassages 0: expected a null snippet, got java.lang.NullPointerException: Cannot invoke "org.apache.lucene.search.uhighlight.Passage.getScore()" because the return value of "java.util.PriorityQueue.peek()" is null
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:AEC4630373516A5C]:0)
        at org.apache.lucene.search.uhighlight.TestZeroMaxPassages.testZeroMaxPassagesWithMatch(TestZeroMaxPassages.java:54)

        Caused by:
        java.lang.NullPointerException: Cannot invoke "org.apache.lucene.search.uhighlight.Passage.getScore()" because the return value of "java.util.PriorityQueue.peek()" is null
            at org.apache.lucene.search.uhighlight.FieldHighlighter.maybeAddPassage(FieldHighlighter.java:212)
            at org.apache.lucene.search.uhighlight.FieldHighlighter.highlightOffsetsEnums(FieldHighlighter.java:194)
            at org.apache.lucene.search.uhighlight.FieldHighlighter.highlightFieldForDoc(FieldHighlighter.java:86)
            at org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFieldsAsObjects(UnifiedHighlighter.java:988)
            at org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFields(UnifiedHighlighter.java:858)
            at org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFields(UnifiedHighlighter.java:836)
            at org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlight(UnifiedHighlighter.java:769)
            at org.apache.lucene.search.uhighlight.TestZeroMaxPassages.testZeroMaxPassagesWithMatch(TestZeroMaxPassages.java:50)

:lucene:highlighter:test (FAILURE): 1 test, 1  failure

# the same failure, 1 of 1, with the standalone runner (seed DEADBEEF): main 645d4fe14a20 with -ea and -da,
# and the 10.6.0, 10.5.2 and 9.12.3 release jars

• Contract. Every overload documents maxPassages only as "The maximum number of top-N ranked passages [per-field] used to form the highlighted snippets" (UnifiedHighlighter.java:757-758, :819-820, :845-846, :881-882, :1089-1090), with no range; the values are never checked (:892-897).
• 0 has a documented meaning nearby. The no-match summary is "the first maxPassages sentences" (:759-761), null for 0 (executed), and getMaxNoHighlightPassages says "If this is 0 then the resulting highlight is null (not formatted)" (:634-638).
• 0 is the only int that ends in a NullPointerException. 1 to Integer.MAX_VALUE - 1 work (Integer.MAX_VALUE fails earlier, with a message-less IllegalArgumentException at FieldHighlighter.java:143, a separate overflow); -1 already fails with an IllegalArgumentException (executed).
• Sibling highlighters. The classic Highlighter coerces the count (Math.max(1, maxNumFragments); // sanity check, Highlighter.java:138); the FastVectorHighlighter rejects only negative counts (BaseFragmentsBuilder.java:145-148) and returns no fragments for 0 (:161); the matchhighlight PassageSelector cuts its result to maxPassages (PassageSelector.java:213) and returns no passages for 0 (executed).
• Tests. The uhighlight tests pass 1, 2, 3 or 10, never 0 (3 from TestUnifiedHighlighterRanking: maxTopN = 3 at :60, highlight(..., n + 1) at :137); the only 0 is the separate withMaxNoHighlightPassages(0) (TestUnifiedHighlighter.java:843). With a check that rejects values below 1, all existing highlighter tests still pass (executed), so none relies on 0.
• History. No commit ever validated maxPassages. The predecessor PostingsHighlighter had the same unguarded check, but its new PriorityQueue<>(n, ...) threw the JDK's message-less IllegalArgumentException for 0 first (executed on 6.2.1, 6.3.0 and 6.6.6): an accident, not a check. The UnifiedHighlighter's Math.min(64, maxPassages + 1) removed the accident and exposed the old check.

The strongest case for "works as designed": UnifiedHighlighter is @lucene.experimental and FieldHighlighter @lucene.internal; "top-N" suggests N > 0; a caller who wants no snippet can leave the field out; no upstream test uses 0; and other Lucene top-N APIs reject 0 ("numHits must be > 0", TopScoreDocCollectorManager.java:64).

Why it fails: that supports at most an explicit IllegalArgumentException with a message (fix B), not a NullPointerException from an empty internal queue that appears only when the document matches. @lucene.experimental allows API changes, not an undocumented crash on an accepted argument, and the module's other highlighters treat 0 as a valid count.

Two options were validated with gradle in a clean worktree at 645d4fe14a20: the minimal test and ./gradlew :lucene:highlighter:test (723 tests: the module's 722 plus the new one). They differ in what 0 means. Maintainers may prefer either; both remove the NullPointerException.

Return new Passage[0] for maxPassages <= 0, before the queue is created (FieldHighlighter.java:137). A matching document asked for 0 passages then takes the same path as a non-matching one: null with default settings; with {0, 1} the first field is null and the second keeps <b>alpha</b> beta. Every value of 1 or more is unchanged.

-    if (off.nextPosition() == false) {
+    if (maxPassages <= 0 || off.nextPosition() == false) {
       return new Passage[0];
     }

Trade-offs

• With withMaxNoHighlightPassages(1), a matching text asked for 0 passages gets the unhighlighted one-sentence summary (alpha beta. gamma alpha.), as a non-matching text already does (executed). If maintainers want null there, the check belongs in highlightFieldForDoc and must keep the summary for non-matching documents.
• Negative values: with <= 0, -1 on a matching document now fails with "Illegal Capacity: -1" from the summary path (as for non-matching documents today) instead of a message-less exception. Writing maxPassages == 0 leaves negative values exactly as today (executed: the minimal test and the five-test suite pass).

What the checks validated

• Gradle, clean worktree at 645d4fe14a20, seed DEADBEEF: the minimal test fails before the change (1 test, 1 failure) and passes after it; :lucene:highlighter:test passes 723 tests, 0 failures.
• On the stock main jars with the class replaced: the minimal test passes with assertions on and off, and an independent five-test suite (each offset source, without a searcher, {title: 0, body: 1}) passes 5 of 5.
• Earlier, a 476-test highlighter run changed only this repro's result.

Throw IllegalArgumentException("maxPassages must be >= 1, got: " + maxPassages) in the FieldHighlighter constructor (the class has only one, so every UnifiedHighlighter entry point, UnifiedHighlighter.java:1152, and any subclass's FieldHighlighter goes through it), or next to the existing argument checks at :892-897.

    if (maxPassages < 1) {
      throw new IllegalArgumentException("maxPassages must be >= 1, got: " + maxPassages);
    }

Trade-offs

Turns a call that works today into an error: 0 on a document without a match returns null now and would throw. Servers that forward a client's count (Solr's hl.snippets=0, by code reading) would answer with an error instead of an empty highlight. Rejecting only negative values and applying the guard for 0 is a middle way (not run).

What the checks validated

Gradle, same setup: the minimal test passes (it accepts the up-front rejection); :lucene:highlighter:test passes 723 tests, 0 failures. 0 and -1 then give "maxPassages must be >= 1" for matching and non-matching documents alike (executed).

• Method: code reading (not a formal method). While modelling how FieldHighlighter forms passages for each offset source, an AI tool noticed that maybeAddPassage treats the queue as full at size() == maxPassages and dereferences peek() without a null check, and asked what happens for 0.
• It wrote a public-API JUnit test, which failed on main with assertions on and off and on 10.5.2 and 9.12.3, and passed with the one-line guard.
• The review's random differential driver drew maxPassages from 1 to 3 only, and the upstream tests pass 1, 2, 3 or 10, which is why neither caught it.