GroupingSearch's cache limit is applied per slice since #16247: with an executor, a query over the limit can be replayed instead of re-run

GroupingSearch.setCaching and setCachingInMB promise one bound for the second-pass cache: it "will not contain more than the maximum specified documents", and if it grows beyond the limit it "is purged and not used in the second pass search". Since #16247 (10.6.0 and main), GroupingSearch passes that limit unchanged to the new CachingCollectorManager, which applies it to each slice. With an IndexSearcher that has an executor, a query is replayed whenever no single slice exceeds the limit, whatever the total, and the caches together can hold up to (number of slices) x the limit. Search results are unchanged. 10.5.2 and any searcher without an executor behave as documented.

• setCachingInMB: "The cache will not grow over a specified limit in MB. ... If the cache grows beyond the specified limit, then the cache is purged and not used in the second pass search." setCaching: "The cache will not contain more than the maximum specified documents", @param maxDocsToCache "The maximum number of documents the cache is allowed to hold" (GroupingSearch.java:240-272). Unchanged since LUCENE-3778 (2012), identical in 10.5.2.

1. One CachingCollector enforces one limit across all segments it sees: each segment gets the remaining budget, the cache is purged when a segment's hits exceed it, and each finished segment's hits are subtracted (CachingCollector.java:80-96, :188-212).
2. GroupingSearch now wraps its first pass in new CachingCollectorManager<>(firstRoundManager, cacheScores, maxCacheRAMMB, maxDocsToCache) and runs searcher.search(query, cachingManager) (GroupingSearch.java:167-173); the second pass replays when cachingManager.isCached(), else re-runs the query (:207-212).
3. IndexSearcher.search(Query, CollectorManager) calls newCollector() once per slice (IndexSearcher.java:750-755, :771). Each call creates a CachingCollector with the full limit (CachingCollectorManager.java:82-91; javadoc: "the maximum number of documents to cache per slice", :64-66).
4. isCached() is true when every per-slice cache is intact (CachingCollectorManager.java:103-111). So the query is replayed exactly when no slice exceeds the limit, and the caches together hold up to slices x limit.
5. Without an executor, IndexSearcher makes one slice of all segments (IndexSearcher.java:218-232), so the old rule holds. Before #16247, GroupingSearch used one CachingCollector with the deprecated search(Query, Collector), which ignores the executor (10.5.2 GroupingSearch.java:151-161, IndexSearcher.java:644-651).

10 segments of 100 matching documents, default slicing: 2 slices of 5 segments. Each slice's cache starts with 600, holds 500 and ends with 100 left; both are intact, so isCached() is true and the second pass replays 1,000 cached documents. On a searcher without an executor the single cache runs out of budget in the seventh segment and is purged, and the query runs again.

setCachingInMB(4.0, true) (package-info.java:89) is 524,288 documents per slice (CachingCollector.java:307-314). A multi-segment default slice holds at most 500,000 documents (IndexSearcher.java:340-385), so on an index whose segments all hold at most 524,288 documents no slice can overflow: every grouped query that finds a group is replayed from caches holding all of its hits, however many it matches (a search that finds no group returns before the second pass, GroupingSearch.java:193-195).

• Opt-in twice. Caching is off by default, and so is the executor. Within that usage default slicing suffices; no unusual parameter is needed.
• Effect. The memory bound the application sized becomes slices x limit per grouped search (still bounded: each slice cache purges at the limit), and a query over the limit is replayed instead of re-run whenever every slice fits. Silent: no error, same results.
• Scale. 16 segments x 125,000 documents, 6 default slices, setCachingInMB(4.0, true): a match-all query was replayed from 2,000,000 cached documents = 16,000,000 bytes of doc IDs and scores at the collector's 8 bytes per hit (computed from the hit count, not measured as heap) against the 4,194,304-byte limit on main and the release candidate; 10.5.2 purged the cache and re-ran the query.
• Releases. 10.6.0 (released 2026-10-29 from RC1) is the first affected release.
• Likelihood: plausible. Concurrent search is common and 10.6.0 makes grouped search concurrent; GroupingSearch caching itself is a little-used opt-in.

# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
    => java.lang.AssertionError: 6 docs match, over the cache limit of 5: the cache must be purged and the query re-run expected:<2> but was:<1>
       org.junit.Assert.fail(Assert.java:89)
       org.junit.Assert.failNotEquals(Assert.java:835)
       org.junit.Assert.assertEquals(Assert.java:647)
       org.apache.lucene.search.grouping.TestGroupingSearchCacheLimit.testCacheLimitAppliesToWholeSearch(TestGroupingSearchCacheLimit.java:63)

• The contract is unchanged. The GroupingSearch setter javadoc (GroupingSearch.java:240-272) comes from LUCENE-3778 (89b55566d83, 2012, first released in 4.0.0), was only reformatted since, and is identical in 10.5.2. It names one limit and the purge rule.
• Behaviour depends on the executor. The finder's test runs the same GroupingSearch over the same reader on a sequential searcher (purged and re-run: 2 executions) and on one with an executor (replayed: 1 execution).
• Never discussed. #16247's description, 1 issue comment, 22 reviews and 39 inline comments say nothing on the limit, slices or memory; its CHANGES entry (CHANGES.txt:543-545, 10.6.0 Improvements) says it parallelizes search; MIGRATE is silent.
• Earlier precedent (as javanna pointed out on #15795). In MemoryAccountingBitsetCollectorManager (merged 2026-06-04), javanna asked: "if it is not shared, the memory limit becomes the number of collectors * the number of slices which is very different from the single threaded search scenario?" gaobinlong, who later wrote #16247, agreed ("we should keep the scope consistent between concurrent search and single threaded search") and shared one tracker. Close precedent in another class, not a decision about GroupingSearch.
• No test pins per-slice semantics for GroupingSearch. TestCachingCollectorManager searches only with limits 0 and Integer.MAX_VALUE (other values appear only in its constructor argument checks); TestGroupingSearch and TestGrouping never assert the limit under slicing.

The strongest case (the intended-behaviour check and the cross-model reviews: moderate): CachingCollectorManager is new, @lucene.experimental, and documents its limits as per slice (CachingCollectorManager.java:24-31, :64-66); the rewritten package-info says it caches documents "per segment slice" (package-info.java:53-58); and #13943 made the top-hits threshold count per slice. GroupingSearch itself (GroupingSearch.java:44) and the IndexSearcher constructors that take an executor (IndexSearcher.java:199, :216) are @lucene.experimental too. So GroupingSearch may simply have inherited per-slice semantics.

Why it fails for GroupingSearch: its own javadoc, the API the user calls, is unchanged and promises one bound; the per-slice wording sits on a different, new class; the package-info sentence describes one cache per slice, not a limit that multiplies; #13943's threshold is a lower bound, which per-slice counting still meets, while this is an upper bound, which it breaks; and #15795 shows the same reviewer and author treating a slice-multiplied limit as a mistake in another class. Experimental status allows a change, but the javadoc still describes the old bound and neither #16247 nor CHANGES says it changed. The intended-behaviour check rated the intended case moderate for CachingCollectorManager and weak for GroupingSearch. Upstream may still keep per-slice semantics and fix only the javadoc.

One code fix was validated in a scratch worktree of 645d4fe14a20 with gradle; a simpler conservative variant was built and run; a javadoc-only change is the option upstream may prefer.

CachingCollectorManager gets a constructor flag limitAppliesToWholeSearch; when set, it holds one AtomicInteger initialised to the limit (MB converted to documents as CachingCollector does), each per-slice CachingCollector takes one unit per collected document and purges itself when the budget is used up, and GroupingSearch passes true. isCached() and replay are unchanged, so one purged slice makes the whole search re-run, as in 10.5.2. The finder's sketch, re-indented for google-java-format.

@@ NoScoreCachingLeafCollector#collect (CachingCollector.java)
   public void collect(int doc) throws IOException {
+    if (docs != null && sharedRemaining != null && sharedRemaining.getAndDecrement() <= 0) {
+      // the limit shared by all caches of one search is used up: purge this cache
+      invalidate();
+    }
@@ GroupingSearch#groupByFieldOrFunction
-              firstRoundManager, cacheScores, maxCacheRAMMB, maxDocsToCache);
+              firstRoundManager, cacheScores, maxCacheRAMMB, maxDocsToCache, true);

Trade-offs

• One atomic decrement per collected document while the cache is live: a shared, contended counter under many threads. Reserving the budget in chunks (for example when a leaf buffer grows) would avoid it; not built.
• It restores the cached-document count and the replay decision, not a strict cap on peak allocation: each leaf buffer starts at up to 128 entries and grows by ArrayUtil.oversize (CachingCollector.java:42, :170, :195-197; refinement by the root-cause reviewer).
• A slice that finished before another used up the budget keeps its arrays until the manager is dropped; they are never replayed.
• API shape: a new public constructor with a boolean. Upstream could instead make the shared budget its only behaviour and drop the "per slice" javadoc (not run).

What the checks validated

• Stock + test: fails with assertions on and off (gradle).
• + fix: the test passes with assertions on and off; core TestCachingCollector, TestCachingCollectorManager, TestIndexSearcher, TestMultiCollector, TestMultiCollectorManager (38 tests, 1 skipped) and the whole lucene/grouping module (15 suites with the new test, 57 tests), 0 failures, assertions on and off.
• :lucene:core:check :lucene:grouping:check -x test (checkGoogleJavaFormat, ecjLint, forbiddenApis) passes after one re-indent.
• Earlier, by the finder: the bounded-exhaustive model-based test with this fix matches the whole-search rule in all 257,152 checks, 0 result mismatches.
• Not run: the full test suite, nightly tests, a benchmark.

GroupingSearch passes limit / max(1, searcher.getSlices().length) to the manager (1 file, +7/-1).

Trade-offs

Smallest change and never over the limit, but conservative: with uneven hits across slices it re-runs queries that fit the whole-search limit (48,208 of 257,152 model-based checks), and setCaching(n) with n below the slice count becomes 0 per slice, so every search with at least one hit re-runs the query (a search with no hits returns before the second pass, GroupingSearch.java:193-195).

What the checks validated

The test, the grouping module (57 tests) and TestCachingCollector* (8 tests) pass, assertions on.

Say in the setCaching / setCachingInMB javadoc that the limit applies to each slice of a concurrent IndexSearcher. Upstream may well choose this.

Trade-offs

Leaves memory at up to slices x limit and the replay decision dependent on the executor; the repro test would then be wrong and should not be added.

• First signal (planning, release-change review of 10.6.0). Reading #16247 against the GroupingSearch javadoc, then a probe: 1,000 matching documents in 4 segments, one segment per slice, a 4-thread executor; setCaching(300, true) and setCaching(999, true) replay on main and branch_10_6 but re-run on 10.5.2, and a sequential searcher re-runs on all three.
• Hunt unit. Reduced it to a public-API JUnit with default slicing, then a bounded-exhaustive differential model-based test of the real jars against two executable models (whole-search rule from the javadoc, per-slice rule): 346 indexes, 4,592 slicings, limits 0..13 as documents and as MB, cacheScores false and true, 257,152 checks. Main and the release candidate follow the per-slice rule in every check and replay over the limit in 33,888; 10.5.2 follows the javadoc in every check; cached results equal uncached ones everywhere.
• SMT model (Z3). Per-slice caching holds at most slices x limit documents, the bound is reached, and one slice equals the whole-search rule.
• The model-based test and the SMT model came after the first signal, so the finding counts as non-formal.