CachingCollector sits between IndexSearcher and the collector it wraps, records each segment's hits, and can replay them into another collector later. It drops the per-segment finish() call in two places.
- For each segment,
IndexSearcherasks the collector for a leaf collector, callscollect(doc)for every match, and then callsfinish()once the segment is done. FacetsCollectorbuffers a segment's matches incollectand adds them togetMatchingDocs()only infinish().CachingCollectorwraps a collector: it forwards each call to it while recording the doc IDs (and, withcacheScores, the scores), soreplay(other)can feed the same hits to another collector without rerunning the query.
- Index 20 documents into one segment, all matching
f:a. Wrap aFacetsCollectorinCachingCollector.create(fc, true, 1000)(cache doc IDs and scores) and search with it. You expect 20 hits ingetMatchingDocs(), as without the cache. - Instead it reports 0.
IndexSearchercallsfinish()on each segment's leaf collector, but the cache's per-segment wrapper overridesfinish()for its own bookkeeping and never forwards it.FacetsCollectoradds a segment's hits togetMatchingDocs()only infinish(). - Replay the cache into a new
FacetsCollector(true): again 0 instead of 20. The replay with cached scores never callsfinish(); the replay of a cache without scores does, and gives 20. - No error with assertions off. With assertions on and two or more segments,
FacetsCollectorthrows anAssertionErrorinstead.CachingCollectorManager(main and 10.6.0, released 2026-10-29) behaves the same. After a cache overflow, later segments do getfinish(), so results turn partial instead of empty.
Applications that put a finish-dependent collector (FacetsCollector, BlockGroupingCollector, the TermGroupFacetCollectorManager collectors, the sandbox HistogramCollector or their own) behind CachingCollector (since 9.8.0) or CachingCollectorManager (10.6.0 and main), or replay a score cache into one. For them it fires on every query. Lucene's GroupingSearch is not affected: its collectors do not implement finish().
Mechanism, terms and numbers522 words
CachingCollector lets an application run a query once, keep its hits and later replay them into another collector. Two calls to LeafCollector#finish(), the hook IndexSearcher calls once a segment is collected, are missing in it, both introduced by #12380 (9.8.0), the change that added the hook: it wrote the wrapper's non-forwarding finish() override and gave only the no-score replay a finish() call. The per-segment wrapper NoScoreCachingLeafCollector extends FilterLeafCollector, whose finish() forwards to the wrapped collector, but overrides finish() for its own bookkeeping without super.finish() (CachingCollector.java:214-221 on main 645d4fe14a20), so the wrapped collector never gets finish() while the cache is live. And the replay with cached scores, ScoreCachingCollector#collect (:144-155), overrides the no-score replay (:98-104) and leaves out its collector.finish() (:103). The same commit moved FacetsCollector's per-segment MatchingDocs into finish() (FacetsCollector.java:116-128), so a FacetsCollector behind the cache, or replayed from a score cache, reports no hits: 0 of 20 in the repro, with no error unless assertions are on and a slice has two or more segments. BlockGroupingCollector loses each segment's last group the same way. It is a regression from 9.7.0, affects every release from 9.8.0 to 10.6.0 (10.6.0 by reading) and main, and the new CachingCollectorManager (in 10.6.0, released 2026-10-29) inherits it. Forwarding the call at both sites (two lines) fixes it; the core caching tests and the grouping and facet modules pass with the fix.
- Wrapped first pass, both
cacheScoresmodes. While the cache is live,getLeafCollectorreturns aNoScoreCachingLeafCollector(or its subclassScoreCachingLeafCollector) around the wrapped leaf collector (CachingCollector.java:80-90). It extendsFilterLeafCollector, whosefinish()isin.finish()(FilterLeafCollector.java:51-54), and itscollectforwards every hit (CachingCollector.java:188-206). But it overridesfinish()with its own bookkeeping only:
@Override
public void finish() {
if (!hasCache()) {
collector.invalidate();
} else {
postCollect();
}
}(CachingCollector.java:214-221). No super.finish(), so the wrapped collector's finish() never runs.
- Replay with cached scores.
replay(other)callscollect(collector, i)per cached segment (:106-118). The no-scorecollectends withcollector.finish()(:103);ScoreCachingCollector#collectoverrides it to replay doc IDs with scores and ends without it (:144-155).
FacetsCollectorrecordsMatchingDocsonly infinish()(FacetsCollector.java:116-128), sogetMatchingDocs()stays empty: 0 of 20 hits. With assertions on and a second segment in the slice,doSetNextReaderfailsassert docsBuilder == null(:107) first.BlockGroupingCollectorprocesses a segment's last group infinish()(BlockGroupingCollector.java:497-502), so that group is lost; with a full queue a returned group can carry another matched document's sort value (executed security probe).- When the cache overflows inside a segment, the following segments get the unwrapped leaf collector (
CachingCollector.java:87-88), whichIndexSearcherfinishes (IndexSearcher.java:862). The result is then partial instead of empty (per-leaffinished=[false, true]). CachingCollectorManagerbuilds oneCachingCollectorper slice (CachingCollectorManager.java:82-91) and inherits both halves; its javadoc example takes the first-pass result directly (:38). Executed through the manager sequentially and with a thread pool; the manager with intra-segment partitions was not run.
Exposure and likelihood173 words
- Opt-in.
CachingCollectorandCachingCollectorManagerare@lucene.experimentalexpert APIs (CachingCollector.java:38,CachingCollectorManager.java:48). Within that usage the default settings suffice, and no scale is needed. - Silent. With assertions off the facet result is simply empty (or holds only the segments after an overflow); nothing is persisted. With assertions on and 2+ segments in a slice, tests see an
AssertionErrorinstead. - Grouping.
BlockGroupingCollectordrops each segment's pending last group; with a full queue a group can come back with another matched document's sort value (executed probe, main and 10.5.2 only: 408 checks on main with assertions off, 433 with them on, 408 on 10.5.2; 0 with the fix). - Releases. Every release from 9.8.0 to 10.6.0 (10.6.0 by reading) and main;
CachingCollectorManager(#16247) first shipped in 10.6.0 (released 2026-10-29). - Likelihood: rare. Lucene's own
GroupingSearchand Solr's grouping cache (by code reading) put no finish-dependent collector behind the cache; saved GitHub code searches found noCachingCollectoruse on the Elasticsearch and OpenSearch default branches. The recorded searches found no report in the three years since 9.8.0.
The test shows that CachingCollectorManager (cacheScores=true) never calls LeafCollector#finish on the wrapped first-pass FacetsCollector, nor on the FacetsCollector the scored cache is replayed into, so both report no hits (1-document index, MatchAllDocsQuery).
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.lucene.facet;
import java.util.List;
import org.apache.lucene.document.Document;
import org.apache.lucene.index.DirectoryReader;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.search.CachingCollectorManager;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.MatchAllDocsQuery;
import org.apache.lucene.store.ByteBuffersDirectory;
import org.apache.lucene.store.Directory;
import org.apache.lucene.tests.util.LuceneTestCase;
public class TestCachingCollectorFinish extends LuceneTestCase {
public void testFacetsCollectorBehindScoreCache() throws Exception {
try (Directory dir = new ByteBuffersDirectory()) {
try (IndexWriter w = new IndexWriter(dir, new IndexWriterConfig())) {
w.addDocument(new Document());
}
try (DirectoryReader reader = DirectoryReader.open(dir)) {
CachingCollectorManager<FacetsCollector, FacetsCollector> caching =
new CachingCollectorManager<>(new FacetsCollectorManager(), true, null, 1000);
FacetsCollector firstPass =
new IndexSearcher(reader).search(new MatchAllDocsQuery(), caching);
FacetsCollector replayed = caching.replay(new FacetsCollectorManager());
// Expected: first pass and scored replay each see the 1 hit; actual: [0, 0] (no finish()).
assertEquals(List.of(1, 1), List.of(hits(firstPass), hits(replayed)));
}
}
}
private static int hits(FacetsCollector fc) {
return fc.getMatchingDocs().stream().mapToInt(FacetsCollector.MatchingDocs::totalHits).sum();
}
}
# in a copy of this finding's repro folder (repro/LFV-180/, which holds src/)
# Lucene checkout at the tested commit, built with: ./gradlew --no-daemon assemble (JDK 25)
LUCENE=/path/to/lucene; V=12.0.0-SNAPSHOT
CP=$LUCENE/lucene/core/build/libs/lucene-core-$V.jar
CP=$CP:$LUCENE/lucene/codecs/build/libs/lucene-codecs-$V.jar
CP=$CP:$LUCENE/lucene/test-framework/build/libs/lucene-test-framework-$V.jar
CP=$CP:$LUCENE/lucene/facet/build/libs/lucene-facet-$V.jar
CP=$CP:randomizedtesting-runner-2.9.2.jar:junit-4.13.2.jar:hamcrest-3.0.jar # test-framework dependencies (Maven Central)
CP=$CP:junit-platform-console-standalone-6.1.3.jar # JUnit Platform console launcher (Maven Central)
javac -d classes -cp "$CP" $(find src -name '*.java')
# run 1: assertions on, seed DEADBEEF
java -ea -esa --add-modules jdk.incubator.vector --enable-native-access=ALL-UNNAMED -Dtests.seed=DEADBEEF \
-cp "classes:$CP" org.junit.platform.console.ConsoleLauncher \
execute --select-class org.apache.lucene.facet.TestCachingCollectorFinish
# the module(s) facet also need their third-party runtime jars
- Expected vs actual
- Expected hits
[first pass, scored replay] = [1, 1], as a plain search gives; actual:[0, 0](expected:<[1, 1]> but was:<[0, 0]>), with-eaand-da.
More files, environment, full output36 words
| lucene | Lucene main e1faeba08bde (2026-11-02) |
|---|---|
| jdk | 25.0.3+9-LTS |
# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
=> java.lang.AssertionError: expected:<[1, 1]> but was:<[0, 0]>
org.junit.Assert.fail(Assert.java:89)
org.junit.Assert.failNotEquals(Assert.java:835)
org.junit.Assert.assertEquals(Assert.java:120)
org.junit.Assert.assertEquals(Assert.java:146)
org.apache.lucene.facet.TestCachingCollectorFinish.testFacetsCollectorBehindScoreCache(TestCachingCollectorFinish.java:45)LeafCollector#finishruns once a segment "has finished collecting successfully" (LeafCollector.java:149-159), andcreate()calls the wrapped collector "the Collector to wrap and delegate calls to".- The replay without scores already calls
finish()(CachingCollector.java:103), added by the same commit (#12380). - Behaviour regression from 9.7.0 (supporting evidence: the hook is new in 9.8.0); delivery that varies with cache overflow cannot be a design.
The full argument151 words
- Contract.
LeafCollector#finish(LeafCollector.java:149-159);CachingCollector.create: "the Collector to wrap and delegate calls to" (CachingCollector.java:301,:320); the leaf wrapper is aFilterLeafCollector, whosefinish()delegates (FilterLeafCollector.java:51-54). - Same commit, inconsistent. #12380 added the forwarding
FilterLeafCollector#finishandcollector.finish()in the no-score replay, but not in the scored replay or the wrapper's override. Its 18 review comments and issue #12375 never discuss CachingCollector's forwarding. - The test framework asserts it.
AssertingCollectorchecks that the previous leaf was finished (AssertingCollector.java:62); wrapping one in aCachingCollectortrips that check (executed). - Precedent. The missing or double
finish()in DrillSidewaysScorer was fixed as a bug (#12640, #12642, #12853); as gsmiller wrote on #12853, "We should consider it a bug for MatchingDocs to not be populated". In 2011 LUCENE-3102 (#4175) fixed "CachingCollector.replay was failing to call setScorer per-segment". - Why tests missed it.
TestCachingCollectorandTestCachingCollectorManageronly use collectors withoutfinish().
The devil's-advocate check136 words
Strongest case (the intended-behaviour check and the works-as-designed cross-model review): the classes are @lucene.experimental; the replay javadoc promises only to replay "the cached doc IDs (and scores)" (CachingCollector.java:345-354); the only in-tree user, GroupingSearch, needs no finish(); and no test caches or replays into a finish-dependent collector.
Why it fails: the no-score replay already calls finish(); the first pass is documented as delegating; delivery that changes with cache overflow is no contract; and experimental status allows API changes, not silently empty results. The cross-model reviews and the intended-behaviour check rated the case weak; the independent reproduction rated it weak for the first pass and weak-to-moderate for the replay. For the scored replay the contract comes from consistency with the no-score replay and the general LeafCollector contract, not from explicit replay wording.
Recommended Forward finish() at both sites
Forward the call at both sites (CachingCollector.java, +3/-1): super.finish() at the start of NoScoreCachingLeafCollector#finish, which then declares IOException, and collector.finish() at the end of ScoreCachingCollector#collect. The test passes; the core caching tests and the grouping and facet modules pass, with assertions on and off.
// ScoreCachingCollector#collect (replay with cached scores)
collector.collect(docs[j]);
}
+ collector.finish();
}
// NoScoreCachingLeafCollector#finish (wrapped first pass)
@Override
- public void finish() {
+ public void finish() throws IOException {
+ super.finish();
if (!hasCache()) {All options and trade-offs3 options ยท 338 words
Two options were built and run in a scratch worktree of 645d4fe14a20; a documentation-only change is not recommended.
super.finish() in the leaf wrapper's finish() (first pass, both cacheScores modes) and collector.finish() after the scored replay loop. The wrapped collector then gets finish() exactly once per segment, whether the cache is live or overflowed (after an overflow IndexSearcher finishes the unwrapped leaf collector, as before).
// ScoreCachingCollector#collect (replay with cached scores)
collector.collect(docs[j]);
}
+ collector.finish();
}
// NoScoreCachingLeafCollector#finish (wrapped first pass)
@Override
- public void finish() {
+ public void finish() throws IOException {
+ super.finish();
if (!hasCache()) {Trade-offs
The override must declare IOException, as FilterLeafCollector#finish already does. Forwarding first means a throwing wrapped finish() skips the cache's bookkeeping for that segment; the search fails with that exception anyway, and putting super.finish() after the bookkeeping works too.
What the checks validated
- Stock + test: fails with assertions on and off; + fix: passes with assertions on and off.
- + fix: core
TestCachingCollector,TestCachingCollectorManager,TestMultiCollector,TestMultiCollectorManager,TestIndexSearcher(38 tests, 1 skipped), the whole grouping module (56 tests) and the whole facet module (307 tests), 0 failures, assertions on and off;:lucene:core:check :lucene:facet:check -x testpasses. - Each line alone leaves the test failing (
[20, 0],[0, 20]). Not run: the full test suite.
Move collector.finish() from NoScoreCachingCollector#collect into the replay loop, right after collect(collector, i) (+3/-2 with the first-pass line).
Trade-offs
One call site, so a future override of collect cannot drop it again; a slightly larger diff that moves an existing line.
What the checks validated
The test, TestCachingCollector* (8 tests) and the grouping module (56 tests) pass, assertions on.
A javadoc note on CachingCollector and CachingCollectorManager.
Trade-offs
Leaves facets behind the cache silently empty and contradicts "the Collector to wrap and delegate calls to". Not recommended.
Solr's grouping cache (by code reading) puts no finish-dependent collector behind CachingCollector, and saved GitHub code searches found no use of it on the Elasticsearch and OpenSearch default branches.
A bounded-exhaustive check: a harness ran the real collector managers, including CachingCollectorManager with FacetsCollectorManager as first and second pass, on every index of 1-3 segments with 1-6 documents under every slicing, and compared each result with a plain search. The hypothesis came from reading #12380; the harness confirmed it in every slicing, then it was reduced to a JUnit.
The method in full113 words
Method: bounded-exhaustive translation validation (semi-formal). While scoping the harness of a hunt unit (concurrent collector managers), the finder read #12380 and suspected that CachingCollector loses LeafCollector#finish. The harness runs the real collector managers on every index of 1-3 segments x 1-6 documents (258 shapes) under every distinct IndexSearcher slicing (3,750), sequentially and with a 3-thread pool, and compares each result with a plain IndexSearcher.search. It witnessed the bug in every slicing: the cached-first-pass failure kinds 360,000 times in the final run. The finder then reduced it by hand to a one-segment public-API JUnit on the stock main jars and a release-compatible variant; this doc's test is a one-method version of those.
No prior report found: 217 queries on 2026-10-26 (GitHub issues and PRs, commits, the dev@, java-user@, issues@ and commits@ lists, the JIRA archive, downstream trackers, the web), refreshed on 2026-10-29. Not fixed upstream: main, branch_10x and branch_10_6 carry both sites, and a scan of the 413 PRs open on 2026-10-26 found none touching them.
Searches5 searches
Related, not reports of this bug: #12380 introduced both halves (it wrote the non-forwarding override and added finish() to the no-score replay only; its reviews never discuss CachingCollector); #12375 lists CachingCollector among the collectors that could use the hook; #16247 added CachingCollectorManager (its reviews never mention finish()); #12640, #12642 and #12853 fixed the same class of bug in DrillSidewaysScorer; #4175 (LUCENE-3102) is the 2011 replay precedent; #10401 (LUCENE-9361, open) proposes removing CachingCollector; open PR #15210 edits the same inner class (a textual conflict, not a fix).
| Where | When | How | Found |
|---|---|---|---|
| GitHub issues and PRs (apache/lucene, apache/solr, Elasticsearch, OpenSearch) | 2026-10-26 | three scripted prior-report searches and 41 GraphQL searches: class and inner-class names, "LeafCollector#finish", FilterLeafCollector finish, "docsBuilder == null", cacheScores, replay finish; bodies and reviews of 17 apache/lucene items read | no report of either half |
| Open apache/lucene PRs | 2026-10-26 | changed files of all 413 open PRs | none touches either site; #15210 edits the same inner class |
| Mailing lists and JIRA archive | 2026-10-26 | 27 list searches (dev@, java-user@, issues@, commits@, general@, Solr lists), 4 JIRA searches | no report; the 2023 dev@ threads on finish() semantics are context |
| Web | 2026-10-26 | 7 web searches | nothing relevant |
| Refresh | 2026-10-29 | upstream branch heads; CachingCollector issues and PRs by update date; three keyword searches over items updated since 2026-10-25 (415 PRs open by then; the changed-file scan was not repeated) | main still 645d4fe14a20, both sites unchanged on branch_10x and branch_10_6, no new item |
Introduced by #12380 (jpountz, merged 2023-06-30, first released in 9.8.0), which added LeafCollector#finish. Every release from 9.8.0 to 10.6.0 (10.6.0 by reading) and main is affected; 9.7.0 and earlier are not. CachingCollectorManager (#16247, new in 10.6.0, released 2026-10-29) inherits it.
Versions tested6 rows
#12380 "Add a post-collection hook to LeafCollector." is 8811f31b9ca6 on main and de07bdd88dd1 on branch_9x, the same patch. It added LeafCollector#finish and the forwarding FilterLeafCollector#finish, replaced CachingCollector's old postCollection() bookkeeping with the non-forwarding NoScoreCachingLeafCollector#finish, added finish() to the no-score replay only, and moved FacetsCollector's MatchingDocs recording into finish(). Before it, FacetsCollector saved its segments without the hook, which works through the cache. #16247 (gaobinlong, merged 2026-07-08) added CachingCollectorManager, listed under 10.6.0.
| Version or commit | Verdict | Result |
|---|---|---|
main 645d4fe14a20 | affected | the test fails, [0, 0], assertions on and off |
| 10.0.0, 10.5.2 | affected | the test fails, [0, 0] (release jars) |
| 9.8.0 to 10.5.2 (23 releases) | affected | the follow-up's history probe gets 12 of 12 checks wrong on each; a release-compatible JUnit fails on all of them |
#12380 commits 8811f31b9ca6, de07bdd88dd1 vs their parents | bisect | built from source: 12 of 12 wrong; the parents 40ee6e583e18, aa0474750584 get every check right |
| 8.0.0 to 9.7.0 (37 releases) | not affected | no finish hook; every probe check right (20 of 20 hits on 9.7.0) |
with the fix (main 645d4fe14a20) | not affected | the test passes; core caching tests, grouping and facet modules pass |
A cross-model refute review (executed), independent verifiers and an adjudication all rate it real; the cross-model reviews call the intended-behaviour case weak. This doc's test was run on main, 10.0.0 and 10.5.2, and with the fix.
Checks and verdicts216 words
Every check re-ran a repro on the stock jars; the reviewer from another model family (OpenAI GPT-5.6) executed its own tests in the refute lens and only read code in the works-as-designed lens.
| Check | Verdict | What it found |
|---|---|---|
| finder (a hunt unit) | candidate | Harness witnessed it in every slicing; JUnit fails 2 of 4 on main (-ea, -da), the release variant fails on 9.8.0, 9.12.3, 10.0.0, 10.5.2 and main and passes on 9.7.0. |
| cross-model refute review | real, high | Own finish-counting tests fail 2 of 4 each for CachingCollector and CachingCollectorManager; pass with the fix. |
| Independent reproducer | real | 13 of 19 own tests fail on main (-ea, -da), 19 of 19 pass with its fix; overflow probe finished=[false, true]; AssertingCollector probe trips. |
| Works-as-designed review | real, high (reading only) | Intended argument weak; a separate run re-ran the repro (2 of 4 fail). |
| Devil's advocate | real | No comment, test, review or list post chose to skip finish(); 9.7.0 passes, 10.5.2 fails. |
| adjudication | confirmed, high | Re-ran the independent reproduction's tests: 13 of 19 fail, 19 of 19 with the fix. |
| this doc | reproduced, fix validated | One-method test on main, 10.0.0, 10.5.2; fix A and B in a scratch worktree. |
The diagram as a still image, generated with AI and checked against the code. It is under 10 MB, the limit for a GitHub attachment.
- PNG 538 KB
Finding record, Lucene and upstream sources, repro files and diagram notes1,547 words
- Registry title
- CachingCollector (and CachingCollectorManager) never calls LeafCollector#finish on the collector it wraps, nor on the second-pass collector when it replays cached scores, so finish-dependent collectors such as FacetsCollector silently report no hits
- Area
- search / CachingCollector first pass and score-caching replay (CachingCollector, CachingCollectorManager)
- Priority
- Low, rank 560: Since 9.8.0 CachingCollector never forwards LeafCollector#finish, so a FacetsCollector behind it silently reports no hits, a natural combination of an experimental API that no known user builds and that -ea tests on several segments catch.
- Impact class
- silent wrong results
- Impact note
- Hits applications that combine CachingCollector (released since 9.8.0, @lucene.experimental) or CachingCollectorManager (new in 10.6.0, released 2026-10-29) with a collector that finalizes its per-leaf results in LeafCollector#finish, either as the wrapped first pass (both cacheScores modes) or as the target of a score-caching replay: with the default configuration and assertions off, FacetsCollector then reports no MatchingDocs at all (0 of 20 hits in the repro) and BlockGroupingCollector drops each leaf's pending last group and can return a wrong group order and sort values (executed by the security probe), while with assertions on and two or more leaves in a slice FacetsCollector throws AssertionError ('assert docsBuilder == null') instead; once the cache overflows, only the leaves after the overflowing one get finish, so results turn partial rather than empty; no scale is needed, nothing is persisted, and the no-score replay and in-tree GroupingSearch (whose collectors do not implement finish) are not affected.
- Summary
- You wrap a FacetsCollector, or any collector that does its per-segment work in LeafCollector#finish, in a CachingCollector or CachingCollectorManager so you can run the query once and replay the hits later, or you replay a score-caching cache into such a collector, and you expect the same facet counts as a plain IndexSearcher search. Instead the collector reports no hits: CachingCollector's per-leaf wrapper overrides finish() for its own bookkeeping and never passes the call on, and the score-caching replay never calls finish() at all, although the no-score replay does. In the repro a query that matches 20 documents gives 0 matching docs, with no error unless assertions are enabled.
- Introduced
- de07bdd88dd1 (branch_9x, released in 9.8.0) and 8811f31b9ca6 (main, released from 10.0.0), the same patch: #12380 'Add a post-collection hook to LeafCollector.' by jpountz, merged 2023-06-30 (closes #12375) [exec: lucene-core and lucene-facet built from source at both commits get 12 of 12 history-probe checks wrong; their parents aa0474750584 (branch_9x) and 40ee6e583e18 (main) get all checks right]; that commit added LeafCollector#finish and the forwarding FilterLeafCollector#finish, replaced CachingCollector's old postCollection() bookkeeping with a NoScoreCachingLeafCollector#finish override that does not call super.finish(), added collector.finish() to the no-score replay only (ScoreCachingCollector#collect, which overrides it, was not touched), and moved FacetsCollector's MatchingDocs recording into finish() [read: git show, git blame, git log -S/-G]; the CachingCollectorManager entry point 3afffeef5a8d (#16247, by gaobinlong, merged 2026-07-08, first released in 10.6.0) inherits it [exec on main] (2023-06-30 (#12380, both commits); 2026-07-08 (#16247, CachingCollectorManager entry point, first released in 10.6.0))
- First affected version
- 9.8.0 [exec] (first main-line release: 10.0.0 [exec]); 9.7.0 is the last release without the bug [exec]: LeafCollector#finish did not exist there and FacetsCollector recorded MatchingDocs without it, which works through CachingCollector; CachingCollectorManager: 10.6.0 (released 2026-10-29; listed under 10.6.0 in CHANGES.txt)
- Affected versions
- every release from 9.8.0 to 10.5.2 (all 23: 9.8.0, 9.9.0-9.9.2, 9.10.0, 9.11.0-9.11.1, 9.12.0-9.12.3, 10.0.0-10.5.2), 10.6.0 (10.6.0 by reading) and main ea57360912ed. [exec] the follow-up's history probe (public-API main() program, lucene-core + lucene-facet, 20 docs as 1 and as 2 segments, 12 bug checks plus controls) gets 12 of 12 checks wrong on each of them with -ea, and also with -da on 9.8.0, 9.12.3, 10.0.0, 10.5.2 and main, and 0 of 12 with the fix sketch shadowed on main; the finder's release-compatible JUnit fails its 2 bug tests on all 23 releases and main (-da too on 9.8.0, 9.12.3, 10.5.2 and main); the CachingCollectorManager JUnit fails 2 of 4 on main with -ea and -da; the independent reproduction's tests fail on 10.5.2, 9.12.3 and 9.8.0, the cross-model review's finish-count test on 10.5.2 and 9.8.0, the intended-behaviour check's re-run on 10.5.2. Not affected [exec]: all 37 releases from 8.0.0 to 9.7.0 (probe; JUnit on 9.1.0-9.7.0) and both parent commits. [read] branch_10x and branch_10_6 (10.6.0, released 2026-10-29) carry both defect sites (their CachingCollector.java differs from main's only by a 'final' modifier); releases before 8.0.0 predate the hook. FacetsCollector and a finish-counting collector were executed on every version; BlockGroupingCollector only on main and 10.5.2 and the TermGroupFacetCollectorManager collector only on main (security probe). Not executed: x86, CachingCollectorManager with intra-segment partitions; the history runs used JDK 25.
- Status
- documented (brief, checks and a per-finding doc exist)
- Verification
- Confirmed by AI-run checks: an independent repro, a cross-model review that agreed, an intended-behaviour check, a prior-report search and a skeptical audit; not yet triaged by a person.
- Found by
- semi-formal: code reading; also reached by exhaustive check
- Upstream and prior reports
- none found (2026-10-26, follow-up prior-report record verdict new: 217 query lines, among them three scripted prior-report searches over GitHub issues/PRs, commits, downstream repositories, the dev@, java-user@, issues@ and commits@ lists, the JIRA archive and the web, 41 GraphQL searches over apache/lucene, apache/solr, elastic/elasticsearch and opensearch-project/OpenSearch, 27 mailing-list full-text searches, 4 JIRA REST searches and 7 web searches, plus the bodies and review comments of 17 apache/lucene items, the file lists of all 413 open apache/lucene PRs and the local git history; none reports either half of the bug); not fixed upstream: on main ea57360912ed, branch_10x and branch_10_6 NoScoreCachingLeafCollector#finish still does not forward and the scored replay still ends without finish(), and no open PR touches either site (#15210 edits NoScoreCachingLeafCollector#setScorer in the same inner class and would conflict textually; #16133 changes FilterLeafCollector#competitiveIterator); related, not reports of this bug: #12380 (introduced it; its 18 review comments and 13 reviews never discuss CachingCollector), #12375 (lists CachingCollector among the collectors that could use the hook), #16247 (CachingCollectorManager; its reviews never mention finish), #4175 / LUCENE-3102 (2011 precedent: replay failing to call setScorer per segment was fixed as a bug), #5436 / LUCENE-4370 (2012 warning that delegating collectors must drive a future finish hook properly, old API), #12640 / #12642 / #12853 (missing or duplicate finish in DrillSidewaysScorer, fixed; gsmiller: 'We should consider it a bug for MatchingDocs to not be populated'), #12558 (what an empty MatchingDocs list did downstream, different cause), the 2023 dev@ threads 'LeafCollector#finish idempotency?' and the Solr 9.8.0 upgrade thread (finish runs once per leaf after successful collection), #10401 / LUCENE-9361 (open proposal to remove CachingCollector); gaps: GitHub's search rate limit was saturated (GraphQL search used instead), list search scores only the first 400 characters of unopened mails, private security reports and chat channels cannot be searched
- Lucene commit tested
- main
645d4fe14a20(commit of 2026-10-28, upstream main on 2026-10-29): the repro test through gradle in a scratch worktree and through the standalone runner, assertions on and off, stock, with the fix, with each half of the fix and with the alternative fix; the 10.0.0 and 10.5.2 release jars (Maven Central, SHA-1 checked). The hunt's earlier runs (finder, verifiers, adjudication, follow-up: 116 history runs over 8.0.0-10.5.2, both #12380 commits and their parents) used mainea57360912ed, where every cited source file exceptCHANGES.txtis identical.
- Finding record
a Lucene main checkout lucene/core/src/java/org/apache/lucene/search/CachingCollector.java:25-38 (class javadoc, @lucene.experimental), :98-104 (no-score replay per-leaf loop, collector.finish() at :103), :145-155 (ScoreCachingCollector#collect, the scored replay, no finish()), :214-221 (NoScoreCachingLeafCollector#finish: own bookkeeping, no super.finish()), :301 and :320 ('the Collector to wrap and delegate calls to'), :346-354 (replay javadoc)a Lucene main checkout lucene/core/src/java/org/apache/lucene/search/FilterLeafCollector.java:30 (@lucene.experimental), :51-54 (finish forwards to in.finish()); LeafCollector.java:149-159 (finish: called once the leaf has finished collecting successfully, 'e.g. to convert facet counts on leaf ordinals to facet counts on global ordinals'); IndexSearcher.java:862 (searchLeaf calls leafCollector.finish())a Lucene main checkout lucene/core/src/java/org/apache/lucene/search/CachingCollectorManager.java:25-41 (wraps a delegate manager, replay 'without re-running the query', example 'R1 firstResult = searcher.search(query, caching)'), :48 (@lucene.experimental), :118-130 (replay into per-slice caches)a Lucene main checkout lucene/facet/src/java/org/apache/lucene/facet/FacetsCollector.java:106-114 (doSetNextReader, 'assert docsBuilder == null' at :107), :116-128 (finish records MatchingDocs)a Lucene main checkout lucene/grouping/src/java/org/apache/lucene/search/grouping/BlockGroupingCollector.java:497-502 (finish processes the leaf's last group); TermGroupFacetCollectorManager.java:240, :346 (leaf finish); lucene/suggest/src/java/org/apache/lucene/search/suggest/document/CompletionScorer.java:70-73 and TopSuggestDocsCollector.java:190-195 (why TopSuggestDocsCollector is out of reach); lucene/test-framework/src/java/org/apache/lucene/tests/search/AssertingCollector.java:62a Lucene main checkout lucene/CHANGES.txt:438 (Lucene 10.6.0 section), :525 (GITHUB#16247 CachingCollectorManager), :548 (GITHUB#16292 TermGroupFacetCollectorManager), :3329 (GITHUB#12380 LeafCollector#finish)
- The finding in 30 seconds
a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/search/CachingCollector.java:80-90, :98-104, :106-118, :144-155, :188-206, :214-221a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/search/FilterLeafCollector.java:51-54; IndexSearcher.java:831-863a Lucene main checkout (645d4fe14a20) lucene/facet/src/java/org/apache/lucene/facet/FacetsCollector.java:77-79, :106-114, :116-128a Lucene main checkout (645d4fe14a20) lucene/grouping/src/java/org/apache/lucene/search/grouping/BlockGroupingCollector.java:497-502a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/search/CachingCollectorManager.java:24-48, :82-91
- Who is affected
a Lucene main checkout (645d4fe14a20) lucene/grouping/src/java/org/apache/lucene/search/grouping/GroupingSearch.java:167-171; TermGroupFacetCollectorManager.java:240, :346; lucene/CHANGES.txt:543, :566
- Why it looks unintended
a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/search/LeafCollector.java:149-159; CachingCollector.java:103, :301, :320, :345-354; FilterLeafCollector.java:51-54a Lucene main checkout (645d4fe14a20) lucene/test-framework/src/java/org/apache/lucene/tests/search/AssertingCollector.java:62
- History
a Lucene main checkout (645d4fe14a20) lucene/CHANGES.txt:3362 (9.8.0 section), :543 (10.6.0 section)
- TestCachingCollectorFinish.java (JUnit test):
repro/LFV-180/src/org/apache/lucene/facet/TestCachingCollectorFinish.java(53 lines)
It is made in parallel with this doc; every on-screen value comes from the code at 645d4fe14a20 or the diagram's executed probe.
assets/LFV-180/bug.png(538 KB)