UnifiedHighlighter with maxPassages = Integer.MAX_VALUE throws a message-less IllegalArgumentException for every document that has a match

The UnifiedHighlighter keeps a field's best maxPassages passages in a java.util.PriorityQueue, created in FieldHighlighter.highlightOffsetsEnums with the initial capacity Math.min(64, maxPassages + 1) (FieldHighlighter.java:141-143 on main 645d4fe14a20). The 64 cap keeps the eagerly allocated initial array small for large limits; the + 1 matches the queue briefly holding one passage more than the limit (why it was written this way is not recorded). For maxPassages = Integer.MAX_VALUE, a conventional "no limit" value (Lucene's own javadocs use it that way, in this module too: MatchRegionRetriever, FastVectorHighlighter), the + 1 wraps to Integer.MIN_VALUE, Math.min returns it, and the JDK constructor throws IllegalArgumentException with no message.

With the stock highlighter that happens for every document whose field has a match; it was executed through highlight, highlightFields and highlightWithoutSearcher, with offsets in postings and with re-analysis, with assertions on and off; in a multi-field call the other fields lose their snippets too. Every limit from 1 to Integer.MAX_VALUE - 1 gives a valid capacity and every one tested works; since 7.3.0, documents without a match are summarized normally. The expression has been there since the UnifiedHighlighter was added (LUCENE-7438, 2016); every release from 6.3.0 to 10.6.0 and main is affected. A one-line overflow-safe clamp fixes it.

• Entry points. highlight(field, query, topDocs, maxPassages) wraps the value in an array and calls highlightFields (UnifiedHighlighter.java:766-770), which ends in highlightFieldsAsObjects (:858). That method checks only "fieldsIn must not be empty" and "invalid number of maxPassagesIn" (:892-897), plus that a searcher is set (:898-902), never the values, and builds one FieldHighlighter per field with its own count (:922-924). highlightWithoutSearcher does the same for one string (:1096-1107).
• No match, no queue. highlightFieldForDoc calls highlightOffsetsEnums (FieldHighlighter.java:86), which returns new Passage[0] when there is no term occurrence (:137-139); the summary path then sizes its list with Math.min(maxPassages, 10) (:112). So Integer.MAX_VALUE works for documents without a match.
• The wrap. With a match, :141-143 builds new PriorityQueue<>(Math.min(64, maxPassages + 1), comparator). 2147483647 + 1 = -2147483648 and Math.min(64, -2147483648) = -2147483648; for 2147483646 the expression is Math.min(64, 2147483647) = 64.
• The throw. JDK 25 PriorityQueue(int, Comparator): if (initialCapacity < 1) throw new IllegalArgumentException();, then new Object[initialCapacity] (PriorityQueue.java:173-175). The queue grows on demand, so 64 is only a starting size.
• The + 1. maybeAddPassage (FieldHighlighter.java:201-224) offers the new passage and polls the lowest when size() > maxPassages (:215-217), so the queue briefly holds maxPassages + 1 entries, which the + 1 matches (the original rationale is not recorded).

On 645d4fe14a20: the minimal test and a probe (highlightWithoutSearcher with 1, 3, 64, 1000, MAX_VALUE - 1, MAX_VALUE; a text without a match; highlightFields with {1, MAX_VALUE}). On ea57360912ed, whose highlighter tree is identical: an independent reproducer's six tests (offsets in postings, re-analysis, highlightWithoutSearcher, {1, MAX_VALUE}, two controls with 3, 63, 64, 1000, MAX_VALUE / 2, MAX_VALUE - 1).

• Who is hit. Code that calls highlight, highlightFields or highlightWithoutSearcher (or a subclass through highlightFieldsAsObjects) with maxPassages = Integer.MAX_VALUE, directly or by forwarding a client's snippet count without an upper bound.
• Default vs opt-in. Default highlighter settings; the trigger is the single argument value Integer.MAX_VALUE. Any field with a match: by reading, every offset source reaches the same line, and offsets from postings and from re-analysis were executed.
• What goes wrong. The call throws on the first matching document, so the request gets no snippets; in a multi-field call all fields lose theirs. The exception has no message and is not the documented one (the only documented IllegalArgumentException is for fields indexed without offsets), so it points the caller at the wrong cause.
• Scale. None needed: one document, one sentence. It failed with every recorded seed, assertions on and off.
• Workaround. Pass Integer.MAX_VALUE - 1 or any realistic bound.
• How likely. Rare: Integer.MAX_VALUE is a conventional "no limit" value (in the same module, MatchRegionRetriever asks for Integer#MAX_VALUE "for all hits to be considered" and FastVectorHighlighter documents its default phrase limit as "unlimited (Integer.MAX_VALUE)"), but the failure is loud on the first matching document, and the recorded searches found no report in ten years.

Copy the test into lucene/highlighter/src/test/org/apache/lucene/search/uhighlight/ of a Lucene checkout and run it with gradle (command below). The IllegalArgumentException is caught only to rethrow it as the cause of an AssertionError that states expected vs actual; the cause keeps the PriorityQueue and FieldHighlighter frames. With the clamp fix it passes unchanged, so it doubles as the regression test.

Before using it in a PR, run ./gradlew tidy: google-java-format re-wraps the comment and the throw line (both over 100 columns); the code does not change, and the re-wrapped file passes checkGoogleJavaFormat.

Executed on main 645d4fe14a20: gradle in a clean scratch worktree (fails), and the standalone runner on the stock main jars with assertions on and off (fails); it also fails on the 10.6.0, 10.5.2 and 9.12.3 release jars.

cp $REPRO/TestUnifiedHighlighterMaxValueMaxPassages.java $LUCENE/lucene/highlighter/src/test/org/apache/lucene/search/uhighlight/
cd $LUCENE && ./gradlew :lucene:highlighter:test --tests org.apache.lucene.search.uhighlight.TestUnifiedHighlighterMaxValueMaxPassages -Ptests.seed=DEADBEEF

# gradle, main 645d4fe14a20, -Ptests.seed=DEADBEEF
TestUnifiedHighlighterMaxValueMaxPassages > testMaxValueMaxPassages FAILED
    java.lang.AssertionError: expected a <b>test</b> for maxPassages = MAX_VALUE, got java.lang.IllegalArgumentException
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:C5AAB44B573F209A]:0)
        at org.apache.lucene.search.uhighlight.TestUnifiedHighlighterMaxValueMaxPassages.testMaxValueMaxPassages(TestUnifiedHighlighterMaxValueMaxPassages.java:38)

        Caused by:
        java.lang.IllegalArgumentException
            at java.base/java.util.PriorityQueue.<init>(PriorityQueue.java:174)
            at org.apache.lucene.search.uhighlight.FieldHighlighter.highlightOffsetsEnums(FieldHighlighter.java:143)
            at org.apache.lucene.search.uhighlight.FieldHighlighter.highlightFieldForDoc(FieldHighlighter.java:86)
            at org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightWithoutSearcher(UnifiedHighlighter.java:1107)
            at org.apache.lucene.search.uhighlight.TestUnifiedHighlighterMaxValueMaxPassages.testMaxValueMaxPassages(TestUnifiedHighlighterMaxValueMaxPassages.java:36)

# standalone runner on the built or released jars, seed DEADBEEF
main 645d4fe14a20       -ea / -da   tests_run=1 failures=1   same cause
10.6.0, 10.5.2, 9.12.3  -ea         tests_run=1 failures=1   same cause

• Contract. Every overload documents maxPassages only as "The maximum number of top-N ranked passages [per-field] used to form the highlighted snippets" (UnifiedHighlighter.java:757-758, :819-820, :845-846, :881-882, :1089-1090), with no range, and nothing validates the values (:892-897). The only documented IllegalArgumentException is for fields indexed without offsets (:744-745, :763-764, :793-794, :825-826, :851-852, :887-888).
• Built for large limits. The cap of 64 keeps a large count from allocating a large queue; every value from 64 to MAX_VALUE - 1 gets capacity 64. The predecessor PostingsHighlighter sized its queue from the raw count and threw OutOfMemoryError ("Requested array size exceeds VM limit") for both MAX_VALUE and MAX_VALUE - 1 (executed on 6.2.1); the cap avoids that for every value except the one where + 1 wraps.
• Intended rejection looks different. setMaxLength and Builder.withMaxLength reject Integer.MAX_VALUE up front with "maxLength must be < Integer.MAX_VALUE" (:192-196, :355-360); their comment names a different + 1 hazard (BreakIterator.preceding(offset+1) and the offsets-queue sentinel). maxPassages never had such a guard.
• Tests. No upstream test passes maxPassages = Integer.MAX_VALUE; the uhighlight tests use withMaxLength(Integer.MAX_VALUE - 1) only.
• History and precedent. LUCENE-7438 (#8490) and its 28 comments never discuss the queue capacity or Integer.MAX_VALUE. LUCENE-10135 (#11172, PR #334, 9.0.0) fixed a similar MAX_VALUE overflow of maxPassageWindow in MatchHighlighter's PassageSelector.

The strongest case for "works as designed": no contract promises Integer.MAX_VALUE; UnifiedHighlighter is @lucene.experimental and FieldHighlighter @lucene.internal; 2^31 - 1 passages is unattainable (by default a stored value is truncated to maxLength, 10,000 characters, when it is loaded, so a field read from the index has at most 10,000 passages); the classic Highlighter in the same module also rejects maxNumFragments MAX_VALUE and MAX_VALUE - 1 ("maxSize must be >= 0 and < 2147483631", executed); and Elasticsearch and OpenSearch pass MAX_VALUE - 1 as their "all fragments" value.

Why it fails: the classic Highlighter has no bounds check of its own (only Math.max(1, maxNumFragments)); its rejection comes from the range check of Lucene's generic org.apache.lucene.util.PriorityQueue (commented "Throw exception to prevent confusing OOME"), a stable limit with a message that names it. Here every value but one works, and that one fails inside a JDK collection, without a message and only for documents with a match. The experimental tag allows API changes, not an undocumented error from an accepted argument, and the downstream MAX_VALUE - 1 is an unexplained value choice, not a documented limit. Verdict: a defect, low severity; clamping or rejecting with a message both fix it.

Two options; the clamp is preferred here for compatibility, and rejecting is also defensible. The clamp was validated on 645d4fe14a20; rejecting the value with a message was not executed. Either way, do not size the queue from maxPassages itself (for example new PriorityQueue<>(maxPassages + 1) or long arithmetic without the cap): java.util.PriorityQueue allocates its initial array eagerly, the predecessor PostingsHighlighter threw OutOfMemoryError that way for both MAX_VALUE and MAX_VALUE - 1, and a downstream engine capped huge fragment counts after the classic Highlighter allocated gigabytes (elastic/elasticsearch#155507). A natural combination: the clamp for this bug, plus a maxPassages < 1 check for the maxPassages = 0 failure (a separate bug, a NullPointerException at FieldHighlighter.java:212).

In FieldHighlighter.highlightOffsetsEnums, compute the initial capacity without overflow. (int) Math.min(64L, maxPassages + 1L) is equivalent.

    PriorityQueue<Passage> passageQueue =
        new PriorityQueue<>(
            maxPassages >= 64 ? 64 : maxPassages + 1, // no int overflow for Integer.MAX_VALUE
            (left, right) -> {

Trade-offs

• The same initial capacity as today for every value below Integer.MAX_VALUE: maxPassages + 1 up to 63, 64 from 64 to Integer.MAX_VALUE - 1; Integer.MAX_VALUE now gets 64 as well. The initial allocation stays bounded.
• It does not change maxPassages = 0 (a separate NullPointerException at FieldHighlighter.java:212) or negative values, which stay invalid and are not claimed: the new expression gives n + 1 for them, like the old one. On main (executed, -1, -5 and Integer.MIN_VALUE), a document with a match gets the same message-less exception at line 143, and a document without a match fails earlier in the summary path (new ArrayList<>(Math.min(maxPassages, 10)), line 112) with "Illegal Capacity: <n>".
• What the validation covers: the positive boundary (Integer.MAX_VALUE) and the existing highlighter suites; no negative input was run with the fix, and below Integer.MAX_VALUE the new expression equals the old one.

What the checks validated

On a clean scratch worktree of 645d4fe14a20 (OpenJDK 25.0.3, seed DEADBEEF): the minimal test fails unmodified and passes with this line; the closest suite TestUnifiedHighlighter (156 tests) and the whole :lucene:highlighter:test (35 suites, 723 tests, including the minimal test) pass with 0 failures (the suite and skip counts read from the JUnit XML of the fact check's identical re-run); checkGoogleJavaFormat passes.

Earlier runs agree: the minimal test passes with the fix shadowed (-ea and -da); the finder's 2 tests and the independent reproducer's 6 tests pass with it, the reproducer's also with the long-arithmetic form.

Check the counts in highlightFieldsAsObjects and highlightWithoutSearcher, as withMaxLength does for maxLength: == Integer.MAX_VALUE covers this bug; < 1 would also turn the maxPassages = 0 failure into a clear error. Not executed.

Trade-offs

Clear error instead of a JDK one, consistent with withMaxLength. But the conventional "no limit" value becomes an error that callers must work around with Integer.MAX_VALUE - 1, while every other large value stays legal.

Method: a static guard inventory, then a guided search (the finder labelled it semi-formal; the last step was a text search, so it sits at the boundary with non-formal), in a hunt unit on accepted-parameter boundaries, run on the stock main jars at ea57360912ed.

• A bytecode pass (the JDK ClassFile API) over 27 core and non-sandbox module jars listed 1,231 public or protected entry points whose code reaches an IllegalArgumentException, with each guard's parameters and constants.
• For UnifiedHighlighter.highlightFields / highlightFieldsAsObjects the only guards are "fieldsIn must not be empty" and "invalid number of maxPassagesIn": the counts themselves are accepted unbounded.
• Searching the accepted-unbounded parameters for + 1 and * 2 arithmetic found Math.min(64, maxPassages + 1); a hand-written public-API JUnit witnessed the failure, later reduced to the single test shown here.