Weight#matches accepts a multi-term interval query with exactly maxExpansions + 1 matching terms, which search rejects

The multi-term interval sources (Intervals.prefix, wildcard, regexp, range, fuzzyTerm, multiterm) take one limit, maxExpansions; prefix, wildcard, range and multiterm document that they throw if the pattern expands to more than maxExpansions terms (the regexp javadoc names the default limit instead, a javadoc slip, and fuzzyTerm has no @throws). MultiTermIntervalsSource checks it in two places. The search path, intervals(), tests ++count > maxExpansions after adding each term and throws at the (maxExpansions + 1)-th term, as documented. The Matches path, matches(), used by Weight#matches and so by highlighters, tests count++ > maxExpansions after adding each term the document contains, so it throws only at the (maxExpansions + 2)-th. A document with exactly maxExpansions + 1 matching terms (3 with limit 2, 129 with the default 128) is rejected by the search and accepted by Weight#matches: UnifiedHighlighter.highlightWithoutSearcher highlights it instead of throwing. Search results, scores and explain never change, and the returned positions are the document's own. Both checks were count++ in 8.0.0; LUCENE-8845 (8.2.0) changed only the search one; #16899 (2026-10-26, on main and branch_11x) changed both throws to TooManyClauses and kept count++. All 61 releases from 8.0.0 to 10.6.0 are affected. The fix is one line, ++count in matches().

• Multi-term interval source: MultiTermIntervalsSource, built by Intervals.prefix, wildcard, regexp, range, fuzzyTerm and multiterm. It walks the segment's terms that the pattern's automaton accepts and combines them into one disjunction.
• maxExpansions: the documented maximum number of terms; the constructor only rejects values above IndexSearcher.getMaxClauseCount() (MultiTermIntervalsSource.java:45-52).
• Weight#matches: returns the match positions of one document (@lucene.experimental, Weight.java:76). The UnifiedHighlighter uses it by default (DEFAULT_ENABLE_WEIGHT_MATCHES = true, UnifiedHighlighter.java:110).

1. Search path, intervals() (MultiTermIntervalsSource.java:57-77): for every accepted term of the segment, subSources.add(...) (:68), then if (++count > maxExpansions) { throw tooManyTerms(); } (:69-71). The count includes the current term, so the (maxExpansions + 1)-th term throws. explain uses this path too (IntervalQuery.java:158-159).
2. Matches path, matches() (:79-148), called from IntervalWeight.matches (IntervalQuery.java:171-175): for every accepted term the document contains (TermIntervalsSource.java:164-169 returns null otherwise), subMatches.add(mi) (:92), then if (count++ > maxExpansions) { throw tooManyTerms(); } (:93-95). The comparison uses the count from before the current term.
3. With limit 2 and the document pa pb pc: after pc the search check compares 3 > 2 and throws; the Matches check compares 2 > 2 and goes on, and returns a Matches with the three positions. A 4th term would compare 3 > 2 and throw.
4. Both throws build the same exception through tooManyTerms() (:150-153) since #16899; only the increment differs.
5. In the usual flow (a scoring or top-hits search, then highlighting its hits with the same query on the same reader) the search has already thrown: it builds the interval iterator of every segment, and a document's matching terms are a subset of the segment's. The bad check shows when matches() runs without such a search. A count alone is no such guarantee: BooleanWeight.count (BooleanWeight.java:167) can answer from the required clauses alone, without building the SHOULD clauses.

Limit 2: 2 terms pass both paths; 3 terms: search throws, Weight#matches returns a Matches; 4 terms throw on both. Default limit 128: 128 pass both; 129: search throws, Matches passes; 130 throw on both. highlightWithoutSearcher on pa pb pc with limit 2 returns <b>pa</b> <b>pb</b> <b>pc</b>; with the fix it throws TooManyClauses.

• Who is hit. Callers of Weight#matches or IntervalsSource#matches on a multi-term interval source that did not first run a scoring or top-hits search of the same query on the same reader. Examples, not a complete list: UnifiedHighlighter.highlightWithoutSearcher (UnifiedHighlighter.java:1096), which uses weight matches by default; a highlighter or other Matches consumer given a highlight query that differs from the search query; MatchRegionRetriever (MatchRegionRetriever.java:405), which calls Weight#matches directly; direct Weight#matches calls. The same holds for a multi-term source nested inside a composite interval source (ordered, unordered, filtered, difference, ...), which pass matches() down to their sub-sources (ConjunctionIntervalsSource.java:75, FilteredIntervalsSource.java:113).
• What it takes. A document whose field holds exactly maxExpansions + 1 distinct matching terms: 3 with limit 2, 129 with the default 128 (Intervals.java:64). One fewer passes both paths, one more throws on both. This is about positive limits; the constructor does not reject maxExpansions < 1.
• What happens. That one call returns a correct Matches (the document's own positions) instead of the documented TooManyClauses (IllegalStateException on 10.x). Code that relies on matches() throwing once a document holds more than maxExpansions matching terms gets a highlight instead. (Search and matches() already differ by design elsewhere: matches() counts only the document's terms, so a document with at most maxExpansions matching terms gets a Matches even where the segment is over the limit. The off-by-one adds the case where the document itself is over the limit.)
• What does not happen. Search results, hit counts, scores and explain never change; nothing over-matches; no hang, and the worst-case work per call does not change: a document within the limit already walks all of the segment's matching terms, opening a postings enum for each (MultiTermIntervalsSource.java:89-97, TermIntervalsSource.java:166). On the trigger input the stock code also walks the rest of them, where the fix stops at the (maxExpansions + 1)-th: in the study's probe (505 matching terms in the segment, limit 2), 506 term-enum steps on stock and 3 with the fix for that document, and at most 506 for any document either way. The returned Matches holds at most one more sub-iterator.
• Default vs opt-in. The default writer, codec, searcher and limit suffice. The intervals package calls its classes experimental and Weight#matches is @lucene.experimental, but the UnifiedHighlighter uses it by default. Every multi-term factory is affected.

Public API only: IndexWriter with the default IndexWriterConfig, ByteBuffersDirectory, TextField, IndexSearcher.createWeight, Weight#matches, IntervalQuery, Intervals.prefix(BytesRef, int); the class sits in the intervals package by Lucene test convention but uses no package-private API. 20 random seeds fail 20 of 20 on main and pass 20 of 20 with the fix.

On 10.x the documented type is IllegalStateException; the test as written fails there too with "no exception was thrown" (10.5.2, assertions on and off). A 10.x backport of the test should expect IllegalStateException.

As the regression test: this same test, for example as a method in TestIntervals.

cp TestMultiTermIntervalsMatchesLimit.java $LUCENE/lucene/queries/src/test/org/apache/lucene/queries/intervals/
cd $LUCENE && ./gradlew :lucene:queries:test --tests org.apache.lucene.queries.intervals.TestMultiTermIntervalsMatchesLimit -Ptests.seed=DEADBEEF
# assertions off: add -Ptests.asserts=false (same result)

TestMultiTermIntervalsMatchesLimit > testMatchesThrowsAtMaxExpansionsPlusOne FAILED
    junit.framework.AssertionFailedError: Expected exception TooManyClauses but no exception was thrown
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:6CD3F92D246F7B78]:0)
        at org.apache.lucene.tests.util.LuceneTestCaseParent.expectThrows(LuceneTestCaseParent.java:1432)
        at org.apache.lucene.tests.util.LuceneTestCaseParent.expectThrows(LuceneTestCaseParent.java:1418)
        at org.apache.lucene.queries.intervals.TestMultiTermIntervalsMatchesLimit.testMatchesThrowsAtMaxExpansionsPlusOne(TestMultiTermIntervalsMatchesLimit.java:47)

# gradle, main 645d4fe14a20, -Ptests.seed=DEADBEEF -Ptests.asserts=false; identical with assertions on

• Contract. Intervals.prefix(BytesRef, int): "@param maxExpansions the maximum number of terms to expand to", "@throws org.apache.lucene.search.IndexSearcher.TooManyClauses if the prefix expands to more than {@code maxExpansions} terms" (Intervals.java:172-174). wildcard (:201-203) and multiterm (:364-367) say the same; so does range (:275-277), though its @throws text says "wildcard glob" (a copy-paste slip). regexp has the same @param (:232), but its @throws reads "more than {@link #DEFAULT_MAX_EXPANSIONS} terms" (:233-234), a javadoc slip: the code checks its maxExpansions argument. fuzzyTerm has no @throws; its @param reads "the maximum number of terms to match" (:329). No exception for the Matches path.
• The sibling check implements it (++count, MultiTermIntervalsSource.java:69), and #16899 gave both paths one message helper so that "both paths produce the same message".
• History. LUCENE-8646 (7d7ab14776b, 8.0.0) wrote both checks as count++. LUCENE-8845 (e8950f4a528, 8.2.0), which made the limit configurable, changed only the search line to ++count, with no CHANGES note on the off-by-one, and tested only intervals(). Asked about it on #9889, the author (romseygeek) said: "This is a general accuracy improvement, which was only picked up when I added specific tests for the expansion cap." Nobody mentioned matches().
• Tests. The only Matches limit assertion, added by #16899 in TestIntervals.testPrefix (TestIntervals.java:984-997), uses limit 1 on a document with three matching terms ("pease", "porridge", "pot"): limit + 2, which throws with either increment.

The strongest case for "intended" (the study's intended-behaviour verifier and the cross-model reviewers): matches() is only a safety net that the normal search-then-highlight flow never reaches; it already counts only one document's terms, so it is more lenient than search by design and one more term changes nothing; and the intervals package and Weight#matches are experimental.

Why it fails: nothing documents an extra term for the Matches path; per-document counting is a separate design choice that the fix leaves alone (with ++count, matches() still counts only the document's terms, now with the documented threshold); experimental means the API may change, not that the documented @throws does not apply; and the project fixed the very same post-increment in the sibling line as "a general accuracy improvement". The argument supports low priority. Verdict: most likely not intended, a defect (a well-supported inference from the javadoc, the sibling line and its history; no maintainer has said either way). Not security: no over-matching and no change to the worst-case work.

Fix A was validated through gradle in a scratch worktree of 645d4fe14a20. B is an alternative that was not run.

Count the current term before comparing, exactly as intervals() does (MultiTermIntervalsSource.java:69). On 10.x the same line is :99.

--- a/lucene/queries/src/java/org/apache/lucene/queries/intervals/MultiTermIntervalsSource.java
+++ b/lucene/queries/src/java/org/apache/lucene/queries/intervals/MultiTermIntervalsSource.java
@@ -90,7 +90,7 @@
       MatchesIterator mi = TermIntervalsSource.matches(te, doc, field);
       if (mi != null) {
         subMatches.add(mi);
-        if (count++ > maxExpansions) {
+        if (++count > maxExpansions) {
           throw tooManyTerms();
         }
       }

Trade-offs

• The two checks then use the same threshold expression (the search still counts the segment's terms, matches() the document's); a document with exactly maxExpansions + 1 matching terms now throws on the Matches path as on the search path.
• Callers that today get a highlight for such a document get the documented exception instead, which is what the limit is for.
• Keeps the per-document counting of matches() unchanged (a separate design point, not part of this bug).

What the checks validated

• Gradle, scratch worktree of 645d4fe14a20, seed DEADBEEF, 2026-11-01: without the fix the repro fails with assertions on and off; with it, it passes with assertions on and off.
• ./gradlew :lucene:queries:test --tests 'org.apache.lucene.queries.intervals.*': 8 classes (including TestIntervals, 43 tests, TestIntervalQuery, 34, and the repro), 118 tests, 0 failures.
• TestUnifiedHighlighterTermIntervals, TestMatchRegionRetriever and TestMatchHighlighter: 144 tests, 0 failures.
• ./gradlew :lucene:queries:check -x test with the fix and the repro as a new test file: passes (google-java-format, ecj lint, forbidden APIs). The full Lucene test suite was not run.
• Earlier: with the fix shadowed into the stock jars, the finder's, the independent verifier's and the cross-model reviewer's own tests pass, and the bounded-exhaustive checker finds 0 failures.

Replace both counters with if (subSources.size() > maxExpansions) in intervals() and if (subMatches.size() > maxExpansions) in matches(), right after the add.

Trade-offs

• Same behaviour as A, and the two copies can no longer drift apart.
• Touches the search path for no behaviour change there; a slightly larger review.

Method: bounded-exhaustive translation validation of the real jar against an independent reference, public API only (study unit on the intervals builder and multi-term sources, main ea57360912ed, 2026-10-26).

• Index: 150 documents in 3 segments over the 14 terms {a,b}^1..3.
• Sources: 1,597 (15 prefix, 84 wildcard, 602 regexp, 360 fuzzyTerm, 536 range), each checked per segment at maxExpansions n - 1, n and n + 1, where n is the size of a reference expansion computed with independent predicates (startsWith, a recursive glob, java.util.regex, Levenshtein or optimal-string-alignment distance, String order): 731,742 checks per seed.
• Result: one failing category, "no throw although the document is over the limit", 9,124 checks with seed 42 and 9,211 with seed 7, every one with exactly maxExpansions + 1 matching terms in the document. 0 failures with the fix; a mutant that loosens the search check by one is caught (4,578 failures), so the checker is not vacuous.
• The finder then wrote the public-API JUnit; the one-test repro was reduced from it.