The grouping second pass never calls LeafCollector#finish on the per-group collectors, so a per-group FacetsCollector silently sees no documents

In a two-pass grouping search, SecondPassGroupingCollector hands each matching document of the top groups to that group's own Collector, created by a GroupReducer. Since 9.8.0 (#12380) a collector learns that a segment is done through LeafCollector#finish(), which IndexSearcher calls once it has collected a segment and whose javadoc names compiling per-segment data, such as facet counts, as its typical use; FacetsCollector records a segment's MatchingDocs only there. The grouping second pass never passes finish() on: GroupReducer.setNextReader replaces each group's leaf collector at the next segment without finishing the old one (GroupReducer.java:84-88 on main 645d4fe14a20), GroupReducer has no finish hook, and SecondPassGroupingCollector does not override finish(), so it keeps LeafCollector's no-op default. So a per-group collector that publishes in finish() ends up empty, with no error: 5 documents in 2 segments, groups a (3) and b (2), both report 0. With assertions on, a per-group FacetsCollector throws AssertionError on the second segment. For that composition it is a regression: the same per-group FacetsCollector worked on 9.7.0, and every release from 9.8.0 to 10.6.0 and main is affected (a collector written against finish() itself never received it through GroupReducer, since the hook only exists from 9.8.0). Stock GroupingSearch, TopGroupsCollector and DistinctValuesCollector are not. Forwarding finish() from SecondPassGroupingCollector to each group's current leaf collector fixes it (+18 lines, validated with the grouping module's tests).

• Second pass: SecondPassGroupingCollector (@lucene.experimental) extends SimpleCollector, so IndexSearcher uses the collector itself as the leaf collector. doSetNextReader calls groupReducer.setNextReader(ctx), and collect routes each document of a top group to groupReducer.collect(value, doc) and returns early for the others (SecondPassGroupingCollector.java:87-102 on main 645d4fe14a20, the call at :95, the early return at :90-92).
• GroupReducer: public abstract class GroupReducer<T, C extends Collector>; "Each group is assigned a Collector returned by newCollector(), and LeafCollector#collect(int) is called for each document that is in a group" (GroupReducer.java:33-40). newCollector() is protected; setNextReader, collect and setScorer are public final. The class has no @lucene.experimental tag; its driver, SecondPassGroupingCollector, has one.
• finish(): "Hook that gets called once the leaf that is associated with this collector has finished collecting successfully, including when a CollectionTerminatedException is thrown. This is typically useful to compile data that has been collected on this leaf, e.g. to convert facet counts on leaf ordinals to facet counts on global ordinals. The default implementation does nothing." (LeafCollector.java:149-159). IndexSearcher.searchLeaf calls it after collecting each segment, also when collection ended with a CollectionTerminatedException or a timeout (IndexSearcher.java:849-862; a segment whose getLeafCollector throws CollectionTerminatedException is skipped, with no leaf collector to finish, :836-840); MultiCollector and FilterLeafCollector forward it.

1. GroupReducer.setNextReader runs collector.leafCollector = collector.collector.getLeafCollector(ctx) for every group (GroupReducer.java:84-88): the previous segment's leaf collector is overwritten without finish().
2. GroupReducer has no finish method, and SecondPassGroupingCollector does not override finish(), so the call from IndexSearcher runs the empty default and reaches no per-group leaf collector, neither between segments nor after the last one.
3. FacetsCollector.doSetNextReader asserts docsBuilder == null and starts a new builder (FacetsCollector.java:106-114); only finish() adds the segment's MatchingDocs (:116-128). Through GroupReducer, with assertions off every group's FacetsCollector has no MatchingDocs; with assertions on the second segment trips the assert.
4. A subclass cannot fix the driver: its methods are final and the per-group holders are private (GroupReducer.java:42, :71-98). A user can only work around it in the collector that the protected newCollector() returns (see the workaround under Suggested fix).

5 documents, groups a, b, a | b, a, in two segments. A collector that counts a segment's hits and publishes them in finish() publishes 5 when IndexSearcher runs it directly, and 0 for group a (expected 3) and 0 for group b (expected 2) through the second pass. An independent probe saw collect() called 3 and 2 times per group and finish() 0 times (2 under IndexSearcher). With a stock FacetsCollector per group: 0 hits instead of 3 with assertions off (totalGroupedHitCount is 5), AssertionError at FacetsCollector.java:107 with assertions on, and 0 on a one-segment index even with assertions on.

• Who is hit. Users of the public, @lucene.experimental SecondPassGroupingCollector constructor with their own GroupReducer whose collector compiles per-segment results in finish(): a FacetsCollector per group, or any collector written against the hook. GroupReducer itself has no experimental tag, but the driver it plugs into does: a public but experimental composition point. Custom reducers are what the class was made for: in LUCENE-7701 (#8752), which created GroupReducer, romseygeek wrote "if you want to create a new type of group summarizer (say some statistics over a group), then you create a GroupReducer implementation".
• Who is not hit. Stock two-pass grouping: the reducers of TopGroupsCollector and DistinctValuesCollector (and so GroupingSearch) use collectors that do nothing in finish().
• Effect. Silent wrong results with assertions off, as in production: every group's finish-dependent collector reports nothing, on every search. With assertions on, a per-group FacetsCollector fails loudly on multi-segment indexes. Nothing is persisted.
• Scale and settings. None needed: one segment is enough. Default IndexWriterConfig, codec and IndexSearcher; the opt-in part is the custom reducer.
• How likely: rare (registry likelihood rare, priority low): a code search for extends GroupReducer on GitHub found 71 files on 2026-10-28 (70 when the fact check repeated it on 2026-10-29), all copies of Lucene's own two reducers.

• Contract. LeafCollector#finish is the once-per-segment lifecycle step of every leaf collector, with facet-count compilation as its named use (LeafCollector.java:149-159). GroupReducer<T, C extends Collector> accepts any collector and gives each group real leaf collectors from getLeafCollector (GroupReducer.java:86). No javadoc, package doc, CHANGES or MIGRATE entry limits per-group collectors to collect().
• History. GroupReducer (2017) predates finish(). #12380 ("Add a post-collection hook to LeafCollector.", jpountz, 2023) added it, made IndexSearcher call it, made MultiCollector and FilterLeafCollector forward it, and moved BlockGroupingCollector, GroupFacetCollector, TermGroupFacetCollector and FacetsCollector onto it, but did not touch GroupReducer or SecondPassGroupingCollector; none of its 18 review comments mentions them. The per-group FacetsCollector works on 9.6.0 and 9.7.0 and fails from 9.8.0; 9.8.0 with only the 9.7.0 FacetsCollector.java swapped in passes.
• Precedent. #12640 ("Ensure DrillSidewaysScorer calls LeafCollector#finish on all sideways-dim FacetsCollectors") and #12853 fixed the same symptom at another fan-out driver; on #12853 gsmiller wrote "We should consider it a bug for MatchingDocs to not be populated".
• Lifecycle elsewhere. On dev@ ("Solr upgrade to Lucene 9.8.0 question", 2023-09-28) jpountz wrote that finish should be called "only once and when collection runs successfully" per segment; the test framework's AssertingCollector asserts that the previous leaf was finished before the next one starts.
• Tests. No Lucene test names GroupReducer or SecondPassGroupingCollector directly. The stock second pass is tested indirectly (TopGroupsCollector and DistinctValuesCollector extend SecondPassGroupingCollector), but no test runs a finish-dependent collector through a custom GroupReducer, so none asserts the behaviour at issue.

The strongest case (the devil's-advocate verifier; rated moderate by the reviews from another model family, OpenAI GPT-5.6): GroupReducer's javadoc promises only collect(); SecondPassGroupingCollector is @lucene.experimental; GroupReducer is a light adapter that also skips setWeight, each collector's own scoreMode(), competitiveIterator() and collect(DocIdStream); both in-tree reducers ignore finish(); and #12375's list of collectors that could use the hook did not include it.

Why it fails: the other skipped calls are separate properties of this adapter, not the mechanism here: setWeight is an optional hook, competitiveIterator() an optional skipping hint, and collect(DocIdStream) defaults to the collect(int) calls each group gets; score needs are declared through the reducer's needsScores() instead of each collector's scoreMode(), so a reducer must set it correctly, and the repro's collector and a default FacetsCollector need no scores. finish() is different: it is where collectors compile their results, and the stock FacetsCollector returns nothing without it. The in-tree reducers explain why it was missed, not that it was meant, and #12375 listed collectors that could use the hook, while GroupReducer is a driver. @lucene.experimental allows API changes, not silent empty results. The 9.7.0-pass / 9.8.0-fail bracket and the DrillSideways fixes show upstream's own view. Verdict: not intended (adjudication: confirmed, high confidence).

One fix was built and validated; three alternatives were considered and not built.

Workaround for users today (not tested): a per-group collector wrapper, returned by the reducer's newCollector(), that finishes its own previous leaf collector in getLeafCollector, plus an explicit finish() on each group's last leaf collector after the search. With concurrent search each slice needs its own second-pass collector and reducer, and the last leaf collector of every group must be finished in every slice; the fix needs no such care.

GroupReducer.finish() calls finish() on each group's current leaf collector and clears the reference; SecondPassGroupingCollector.finish() calls it (2 files, +18 lines with a javadoc; the same code as the finder's sketch). On the normal path each per-group leaf collector then gets what IndexSearcher gives a top-level one: one finish() per segment, after collection, also when IndexSearcher caught a CollectionTerminatedException or a timeout during scoring, as MultiCollector and FilterLeafCollector already do. Exceptions while the per-group leaf collectors are set up or finished are not covered (see the trade-offs).

// GroupReducer
public final void finish() throws IOException {
  for (GroupCollector<C> collector : groups.values()) {
    if (collector.leafCollector != null) {
      collector.leafCollector.finish();
      collector.leafCollector = null;
    }
  }
}
// SecondPassGroupingCollector
@Override
public void finish() throws IOException {
  groupReducer.finish();
}

Trade-offs

• It adds a public final method to a public abstract class: a user subclass that already declares a finish() method would stop compiling. The package-private variant below avoids that.
• Open PR #16548 (gaobinlong, stale), the only open PR editing GroupReducer.java, catches CollectionTerminatedException per group and drops that group's leaf collector without finish(). If both land, the merged code should finish such a collector before dropping it, since the finish javadoc covers CollectionTerminatedException.
• Exceptions need a decision. If one group's getLeafCollector throws inside setNextReader, the groups before it already hold new leaf collectors that are never finished: a CollectionTerminatedException there makes IndexSearcher skip the segment (IndexSearcher.java:836-840) and the next setNextReader overwrites them; any other exception fails the search, where no finish() is due. A nested finish() that throws stops the loop and fails the search, as in MultiCollector.finish(). The fix leaves both cases as they are. A group's CollectionTerminatedException during collect() is a separate issue (LFV-883).

What the checks validated

• Scratch worktree of 645d4fe14a20, gradle, OpenJDK 25: the repro fails on the stock tree and passes with the fix, with assertions on and off.
• With the fix and the repro test, the whole lucene/grouping module passes with assertions on and off: 15 suites (14 upstream + the test), 57 tests, 0 failures, including TestGrouping, TestGroupingSearch, TestDistinctValuesCollector, TestTopGroups and TestBlockGrouping.
• :lucene:grouping:check -x test (checkGoogleJavaFormat, ecjLint, forbiddenApis) passes.
• Earlier, by the hunt: the finder's FacetsCollector test and the independent reproducer's 4 tests pass with the same code with assertions on and off; the devil's advocate's probe (2 finish() calls per group for 2 segments) and the root-cause test on 9.8.0 pass with assertions on (not run with them off on the fix).
• Later, by the fact check: the repro with the fix with assertions on and off, and the finder's test with the fix with assertions on.
• Not run: the full Lucene test suite, nightly tests, a benchmark (the change adds one loop over the groups per segment).

void finish() (or another name) without public, called only by SecondPassGroupingCollector in the same package.

Trade-offs

No API change: a package-private method is not visible to subclasses in other packages, so an existing user finish() keeps compiling (checked with javac on two toy classes). But setNextReader, collect and setScorer are public, which suggests GroupReducer may be driven from outside the package, and such a driver could not finish the collectors. Not built.

Call finish() on each group's old leaf collector before replacing it.

Trade-offs

Incomplete on its own: without an end-of-segment hook the last segment is never finished, and the others are finished late, when the next segment starts. Not built.

Say in GroupReducer's javadoc that per-group collectors get no finish().

Trade-offs

Cheap, but leaves a per-group FacetsCollector, which worked before 9.8.0, broken. Not built.

Method: semi-formal translation validation (the hunt's hunt unit for a sweep of public facet, join and grouping APIs, 2026-10-28, main a9779ef3c193).

• Lead. While modelling the second-pass fan-out of GroupReducer against the LeafCollector driver protocol (code reading; no model artifact was kept), the finder saw that the per-group leaf collectors are replaced on every segment without finish() and that SecondPassGroupingCollector forwards no finish().
• Witness. A public-API JUnit on the stock main jars that gives each group a stock FacetsCollector failed with assertions on and off; it was run on releases back to 9.6.0 and with the fix sketch.
• Checks. An independent reproducer wrote its own 4-test class; a devil's advocate built the strongest intended-behaviour case and its own probe; reviews by another model family (OpenAI GPT-5.6) could only read code; an adjudication re-ran the tests. This doc reduced the repro to one grouping-only method.