All findings · Lucene finding · LFV-961 · grouping-package-doc-topngroups-includes-offset

The grouping package overview says groupOffset 3 with topNGroups 10 returns 7 groups, but the collector managers it names return 10

The grouping package overview says groupOffset 3 with topNGroups 10 gives 7 groups, but the collector managers it names since 10.6.0 return 10, because for them topNGroups is the page size after the offset. The code is right; paging written from that sentence gets oversized, overlapping pages.

Summary

Priority
Low, rank 1235This grouping doc defect, mostly new in 10.6.0, can silently lead users who follow the package prose into oversized, overlapping pages.
Impact
minor
Since
10.6.0 (#16385) for the direct contradiction; the sentence dates from 3.2 (2011)introduced 2026-08-12
Affected
10.6.0 and main; the managers behave this way since 10.5.0
Status
documented
Found by
non-formal: noticed while modelling grouping offset + limit sums, then a JUnit test

AI-assisted page: this finding was first signalled by code reading or testing and then checked with AI tools; the checks it went through, and what each found, are under Prior reports, history and checks. Not yet triaged by a human. Snapshot of 2026-11-06.

The finding in 30 seconds

Both managers give each collector a pool of groupOffset + topNGroups groups, then merge, skip groupOffset groups and return up to topNGroups. So topNGroups is the page size. The inputs list describes the plain collectors, whose topNGroups includes the offset; since 10.6.0 the page names the managers and no plain collector, and their own @param texts do not correct it.

Background: paging grouped results

  1. A grouped search ranks groups, for example all books by one author, by groupSort. To show page 2 at 5 groups per page you skip 5 groups (groupOffset) and ask for the next 5.
  2. The plain first-pass collector FirstPassGroupingCollector(selector, sort, topNGroups) keeps topNGroups groups in total, and getTopGroups(groupOffset) drops the first groupOffset of them: offset 3 of 10 leaves 7.
  3. Since 10.5.0 the module also has collector managers for concurrent search, FirstPassGroupingCollectorManager(selector, sort, groupOffset, topNGroups) and BlockGroupingCollectorManager(sort, groupOffset, topNGroups, ...). Since 10.6.0 the package overview names these and no plain collector.
Summary frame titled 'Lucene's grouping docs promise 7 groups; the classes they name return 10'. Three strips over 15 one-document groups g00 to g14 ranked by index order, groupOffset 3 and topNGroups 10: the page says 7 groups back; FirstPassGroupingCollectorManager and BlockGroupingCollectorManager return 10, g03 to g12; the plain FirstPassGroupingCollector and BlockGroupingCollector with getTopGroups(3) return 7. Three boxes: why (each manager keeps groupOffset + topNGroups groups per collector, skips groupOffset and returns up to topNGroups, as GroupingSearch does), since when and what it costs (the inputs list is unchanged since 3.2; #16385 in 10.6.0 made the page name only the managers; paging 5 per page from the sentence repeats g10 to g14), and the documentation-only fix.
15 groups g00 to g14 with groupOffset 3 and topNGroups 10: the page promises 7 groups, the two managers return 10 (g03 to g12), and only the plain collectors return 7. Select the still to open it at full size. Generated with AI, checked against the code.

The finding, step by step

  1. You page grouped results with the first-pass class the grouping overview names, FirstPassGroupingCollectorManager. Its inputs list says topNGroups is "how many top groups to keep" and that groupOffset 3 "means you'll get 7 groups back (assuming topNGroups is 10)".
  2. On an index with 13 groups you pass groupOffset 3 and topNGroups 10, and you expect 7 groups: ranks 4 to 10.
  3. You get 10 groups: ranks 4 to 13. BlockGroupingCollectorManager and GroupingSearch.search(searcher, query, 3, 10) also return 10; only the plain collectors return 7: FirstPassGroupingCollector with getTopGroups(3), and BlockGroupingCollector with topNGroups 10 and groupOffset 3.
  4. Paging written from the sentence, 5 groups per page (groupOffset 5p, topNGroups 5p+5), over 15 groups g00 to g14: page 1 returns g05 to g14 (10 groups) and page 2 repeats g10 to g14.
  5. Why: each manager keeps groupOffset + topNGroups groups per collector, then skips groupOffset and returns up to topNGroups, as GroupingSearch documents. The sentence describes the plain collectors, which the page stopped naming in 10.6.0 (#16385).

Who is affected

Developers who page grouped results with FirstPassGroupingCollectorManager or BlockGroupingCollectorManager (experimental, since 10.5.0) and take topNGroups from the package overview, its most explicit definition (the managers' @param texts read the same way). Their pages are larger than the page size and repeat groups of the next page; every group still matches the query. GroupingSearch users are not affected.

Mechanism, terms and numbers566 words

The lucene-grouping package overview defines the two paging inputs of a grouped search: topNGroups is "how many top groups to keep. For example, 10 means the top 10 groups are computed", and groupOffset 3 "means you'll get 7 groups back (assuming topNGroups is 10). This is useful for paging, where you might show 5 groups per page". Since #16385 (10.6.0) the same page names FirstPassGroupingCollectorManager as the first pass and builds BlockGroupingCollectorManager(groupSort, groupOffset, topNGroups, ...). Both managers, like GroupingSearch, treat topNGroups as the number of groups to return after the offset, so (3, 10) returns 10 groups, ranks 4 to 13. The "7 groups" rule holds only for the plain FirstPassGroupingCollector and BlockGroupingCollector, whose topNGroups includes the offset, and the 10.6 page names neither. The code is correct and intended; the text needs fixing. The bullet has also disagreed with the page's own example code since 3.2.

The text (main 645d4fe14a20, package-info.java)

  • :37-41: "topNGroups: how many top groups to keep. For example, 10 means the top 10 groups are computed." / "groupOffset: which "slice" of top groups you want to retrieve. For example, 3 means you'll get 7 groups back (assuming topNGroups is 10). This is useful for paging, where you might show 5 groups per page."
  • :49-52: "the first pass (FirstPassGroupingCollectorManager) gathers the top groups, and the second pass (TopGroupsCollectorManager) gathers documents within those groups".
  • :129-137: new BlockGroupingCollectorManager<>(groupSort, groupOffset, topNGroups, needsScores, lastDocPerGroup, withinGroupSort, docOffset, docsPerGroup). No plain collector is named anywhere on the page.

The code

// FirstPassGroupingCollectorManager.java:106-125
public FirstPassGroupingCollector<T> newCollector() throws IOException {
  return new FirstPassGroupingCollector<>(
      groupSelectorFactory.get(), groupSort, groupOffset + topNGroups, ignoreDocsWithoutGroupField);
}
... reduce(...): allGroups.add(collector.getTopGroups(0)) for each collector, then
return SearchGroup.merge(allGroups, groupOffset, topNGroups, groupSort);
  • SearchGroup.merge keeps a queue of offset + topN (SearchGroup.java:281), skips the first offset groups (if (count++ >= offset), :304) and stops when it holds topN (:309-310).
  • BlockGroupingCollectorManager does the same: new BlockGroupingCollector(groupSort, groupOffset + topNGroups, ...) (BlockGroupingCollectorManager.java:141-144), then TopGroups.mergeBlockGroups(shardGroupsList, groupSort, groupOffset, topNGroups, withinGroupSort) (:158-159).
  • GroupingSearch documents this meaning: "@param groupLimit The number of groups to return from the specified group offset" (GroupingSearch.java:123), and passes groupOffset, groupLimit to the first-pass manager (:148).
  • Where "7" is right: the plain collectors take a pool size that includes the offset, "@param topNGroups How many top groups to keep" (FirstPassGroupingCollector.java:72, :86; BlockGroupingCollector.java:204), and the offset goes to getTopGroups (FirstPassGroupingCollector.java:136; BlockGroupingCollector.java:260-261).
  • The managers' own javadoc reads like the bullet: "@param groupOffset The offset in the collected groups" and "@param topNGroups the number of top groups to collect" (FirstPassGroupingCollectorManager.java:66-67, :82-83; the first is word for word FirstPassGroupingCollector.getTopGroups's @param, :133), and "the number of top groups to collect" (BlockGroupingCollectorManager.java:88). Only GroupingSearch is unambiguous.

The numbers

13 one-document groups g0 to g12 by index order, (groupOffset 3, topNGroups 10): FirstPassGroupingCollectorManager returns 10 groups, g3 to g12 (the minimal test; the first group checked in this doc's fixed-text run). 15 groups g00 to g14 (the independent reproduction's test): the manager returns g03 to g12, BlockGroupingCollectorManager 10 groups, GroupingSearch.search(searcher, q, 3, 10) 10 groups from g03, and the plain FirstPassGroupingCollector(selector, sort, 10).getTopGroups(3) 7. Paging from the text, groupOffset 5p and topNGroups 5p+5: page0=[g00..g04], page1=[g05..g14], page2=[g10..g14].

Exposure and likelihood205 words
  • Who: code written against the two collector managers from the overview's inputs list. Users of GroupingSearch, and anyone who passes the page size as topNGroups (the meaning GroupingSearch documents for its groupLimit), get the intended page size.
  • Default, not a special setting: no index size, codec or flag is involved. The minimal repro uses 13 one-document groups, and 11 already show the mismatch: with (3, 10) the first-pass manager returns 6, 7, 8, 9 and 10 groups for 9, 10, 11, 12 and 13 groups, so from 11 groups on it returns more than the sentence's 7. Since 10.6.0 the page names the managers and no plain collector, so the overview now leads its readers to them.
  • Effect: silent. No exception; pages look plausible. Nothing excluded or deleted appears: every group comes from the same query, filters and reader. The first pass also does somewhat more work than meant (each collector keeps about 2 x groupOffset + pageSize groups instead of groupOffset + pageSize).
  • Likelihood: plausible but limited. The managers are new (10.5.0) and @lucene.experimental, most users go through GroupingSearch, and no one has reported the mismatch in the roughly 15 years the bullet has disagreed with the page's examples.

Minimal repro

One JUnit test, public API only: 13 one-document groups, a default IndexWriterConfig, ByteBuffersDirectory, and FirstPassGroupingCollectorManager with the overview's own numbers, groupOffset 3 and topNGroups 10. It asserts the documented 7 groups. Deterministic, single thread, the same result with assertions on and off.

TestGroupingPackageDocOffset.java56 lines
/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.apache.lucene.search.grouping;

import java.util.Collection;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.SortedDocValuesField;
import org.apache.lucene.index.DirectoryReader;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.MatchAllDocsQuery;
import org.apache.lucene.search.Sort;
import org.apache.lucene.store.ByteBuffersDirectory;
import org.apache.lucene.store.Directory;
import org.apache.lucene.tests.util.LuceneTestCase;
import org.apache.lucene.util.BytesRef;

public class TestGroupingPackageDocOffset extends LuceneTestCase {

  public void testOffsetThreeOfTopTenGivesSevenGroups() throws Exception {
    try (Directory dir = new ByteBuffersDirectory()) {
      try (IndexWriter w = new IndexWriter(dir, new IndexWriterConfig())) {
        for (int g = 0; g < 13; g++) {
          Document doc = new Document();
          doc.add(new SortedDocValuesField("g", new BytesRef("g" + g)));
          w.addDocument(doc);
        }
      }
      try (DirectoryReader r = DirectoryReader.open(dir)) {
        Collection<SearchGroup<BytesRef>> groups =
            new IndexSearcher(r)
                .search(
                    MatchAllDocsQuery.INSTANCE,
                    new FirstPassGroupingCollectorManager<>(
                        () -> new TermGroupSelector("g"), Sort.INDEXORDER, 3, 10));
        // package-info: groupOffset 3 "means you'll get 7 groups back (assuming topNGroups is 10)"; actual 10
        assertEquals(7, groups.size());
      }
    }
  }
}
Command
cp $REPRO/TestGroupingPackageDocOffset.java $LUCENE/lucene/grouping/src/test/org/apache/lucene/search/grouping/
cd $LUCENE
./gradlew :lucene:grouping:test --tests org.apache.lucene.search.grouping.TestGroupingPackageDocOffset -Ptests.seed=DEADBEEF -Ptests.asserts=true
# assertions off: -Ptests.asserts=false (same failure)
Expected
7 groups, as the package overview promises for groupOffset 3 and topNGroups 10.
Observed
AssertionError: expected:<7> but was:<10>: the manager returns the groups ranked 4 to 13.

All files of this repro: repro/LFV-961/

More files, environment, full output224 words
  • Smallest input. 13 groups are the fewest for which the manager returns its full topNGroups (10); 11 or 12 would also differ from 7 but not show the full 10.
  • Releases. The same test fails the same way on the 10.6.0 and 10.5.0 release jars, assertions on and off. It needs the managers, so it does not apply before 10.5.0, and on 10.5.x the page did not name them yet.
  • Not a regression test. It asserts the documented promise, so it is evidence for a doc fix: with the fix it would be dropped, or kept with its one assertion changed to the corrected text's example (10 groups, the first being g3), which this doc ran.

Environment

lucenemain 645d4fe14a20 (11.0.0-SNAPSHOT); also fails on the 10.6.0 and 10.5.0 release jars
jdkOpenJDK 25.0.3
osLinux aarch64
Command (environment)
cp $REPRO/TestGroupingPackageDocOffset.java $LUCENE/lucene/grouping/src/test/org/apache/lucene/search/grouping/
cd $LUCENE
./gradlew :lucene:grouping:test --tests org.apache.lucene.search.grouping.TestGroupingPackageDocOffset -Ptests.seed=DEADBEEF -Ptests.asserts=true
# assertions off: -Ptests.asserts=false (same failure)

Observed output

# gradle, main 645d4fe14a20, seed DEADBEEF, -Ptests.asserts=true
TestGroupingPackageDocOffset > testOffsetThreeOfTopTenGivesSevenGroups FAILED
    java.lang.AssertionError: expected:<7> but was:<10>
        at org.apache.lucene.search.grouping.TestGroupingPackageDocOffset.testOffsetThreeOfTopTenGivesSevenGroups(TestGroupingPackageDocOffset.java:52)
BUILD FAILED
# -Ptests.asserts=false: the same AssertionError
# release jars 10.6.0 and 10.5.0, assertions on and off: 1 run, 1 failed, expected:<7> but was:<10>
# the same test with its assertion changed to the fixed text (10 groups, first g3), fix A applied: passes, assertions on and off

Why it looks unintended

  • The page contradicts itself: its own examples always passed groupOffset + topNGroups to the plain collectors, which returns 10 groups, not 7.
  • The managers' meaning was chosen in the #15936 review to match GroupingSearch, and TestGrouping checks it.
  • #16385 renamed the classes on the page but kept the inputs list; nothing endorses it for the managers.
The full argument158 words
  • Self-contradiction since 2011: LUCENE-1421's two-pass example passed groupOffset+topNGroups to FirstPassGroupingCollector, and LUCENE-3129's Block example new BlockGroupingCollector(groupSort, groupOffset+topNGroups, ...) with getTopGroups(withinGroupSort, groupOffset, ...). Run with topNGroups 10 and groupOffset 3, that example returns 10 groups on 9.12.3, 10.4.0, 10.5.2, 10.6.0 and main.
  • The code is intended: in the #15936 review a reviewer asked whether the Block manager should "also use groupOffset", and the author changed it to groupOffset+topNGroups to match GroupingSearch. TestGrouping's oracle computes limit = Math.min(groupOffset + topNGroups, groups.size()) and returns limit - groupOffset groups (TestGrouping.java:676, :680), and passes groupOffset, topNGroups straight to both managers (:1110-1112, :1300-1309).
  • Nothing endorses the bullet for the managers: #16385 rewrote the class names and the example at a reviewer's request and left the inputs list; no review comment discusses it, no test uses (3, 10) or asserts the bullet's 7 groups (the assertEquals(7, ...) calls in TestGrouping.java:173-174 and TestGroupingSearch.java:125-126 check hit counts).
The devil's-advocate check139 words

Strongest case (a separate intended-behaviour verifier and a cross-model reviewer, OpenAI GPT-5.6, works-as-designed lens): the inputs list is a 2011 conceptual description of the low-level collectors; "how many top groups to keep" is their @param word for word, and for them "7 groups" is exactly right; the managers' meaning was decided in review and is tested; GroupingSearch documents it; the looseness is 15 years old and unreported.

Why it fails: all of that shows the code is right, which is the claim. The 10.6 page names no plain collector; it names the managers and shows a manager example, and for every paging API it names (both managers and GroupingSearch) its example "3 means you'll get 7 groups back" is false. The managers' own @param texts read like the bullet, so a manager user has nothing that corrects it.

Suggested fix

Recommended A. Rewrite the topNGroups and groupOffset bullets, naming both plain collectors (recommended)

Rewrite the two bullets in the package overview: topNGroups is how many groups to return counted from groupOffset, so 3 with 10 returns the groups ranked 4 to 13, as for GroupingSearch and the managers; add that the plain FirstPassGroupingCollector and BlockGroupingCollector take a count that includes the offset. Text only; the javadoc build and grouping tests pass.

 *   <li><code>topNGroups</code>: how many groups to return, counted from <code>groupOffset</code>.
 *       For example, 10 means up to 10 groups are returned.
 *   <li><code>groupOffset</code>: which "slice" of top groups you want to retrieve. For example, 3
 *       with <code>topNGroups</code> 10 returns the groups ranked 4 to 13. This is useful for
 *       paging, where you might show 5 groups per page: page <code>p</code> (counting from 0) uses
 *       <code>groupOffset</code> 5*p and <code>topNGroups</code> 5. This is how {@link
 *       org.apache.lucene.search.grouping.GroupingSearch} and the collector managers below read
 *       these inputs. The low-level collectors ({@link
 *       org.apache.lucene.search.grouping.FirstPassGroupingCollector}, {@link
 *       org.apache.lucene.search.grouping.BlockGroupingCollector}) instead keep a number of groups
 *       that includes the offset: create them with <code>groupOffset + topNGroups</code> and pass
 *       <code>groupOffset</code> to <code>getTopGroups</code>.
All options and trade-offs3 options ยท 485 words

The behaviour is intended, so every option is a text change. Option A was applied in a scratch worktree of 645d4fe14a20; option B was built on top of it.

A. Rewrite the topNGroups and groupOffset bullets, naming both plain collectors (recommended) (validated by the checks)

Replace package-info.java:37-41 with the text below. A shorter variant keeps only the first two bullets plus one sentence on the plain collectors.

 *   <li><code>topNGroups</code>: how many groups to return, counted from <code>groupOffset</code>.
 *       For example, 10 means up to 10 groups are returned.
 *   <li><code>groupOffset</code>: which "slice" of top groups you want to retrieve. For example, 3
 *       with <code>topNGroups</code> 10 returns the groups ranked 4 to 13. This is useful for
 *       paging, where you might show 5 groups per page: page <code>p</code> (counting from 0) uses
 *       <code>groupOffset</code> 5*p and <code>topNGroups</code> 5. This is how {@link
 *       org.apache.lucene.search.grouping.GroupingSearch} and the collector managers below read
 *       these inputs. The low-level collectors ({@link
 *       org.apache.lucene.search.grouping.FirstPassGroupingCollector}, {@link
 *       org.apache.lucene.search.grouping.BlockGroupingCollector}) instead keep a number of groups
 *       that includes the offset: create them with <code>groupOffset + topNGroups</code> and pass
 *       <code>groupOffset</code> to <code>getTopGroups</code>.

Trade-offs

Text only. Keeps the page's paging example and makes it concrete; names both plain collectors, as the adjudication and the cross-model works-as-designed review asked (the hunt's first diff named only the first-pass one). "The collector managers below" means the two that take these inputs (FirstPassGroupingCollectorManager, BlockGroupingCollectorManager); the other managers on the page (TopGroupsCollectorManager, AllGroupHeadsCollectorManager, TermGroupFacetCollectorManager) take neither.

What the checks validated

  • :lucene:grouping:checkGoogleJavaFormat and :lucene:grouping:renderJavadoc pass; the rendered package summary shows the new text with working links to GroupingSearch, FirstPassGroupingCollector and BlockGroupingCollector.
  • The minimal test fails before the fix with assertions on and off; the same test with its one assertion changed to the new text's example (10 groups, first g3, ranks 4 to 13) passes with assertions on and off.
  • The whole :lucene:grouping:test with assertions on: 15 classes, 57 tests, 0 failures (TestGrouping 7/7); TestGrouping again with assertions off: 0 failures. Seed DEADBEEF, OpenJDK 25.0.3.

B. Option A plus clearer @param texts on the two managers (optional) (validated by the checks)

FirstPassGroupingCollectorManager (both constructors): @param groupOffset "how many top groups to skip", @param topNGroups "how many groups to return after the skipped ones"; BlockGroupingCollectorManager: @param topNGroups "how many groups to return, starting at groupOffset".

Trade-offs

Removes the ambiguity at the source a manager user reads first; five @param lines reworded in three constructor javadocs (no lines added).

What the checks validated

With A applied: checkGoogleJavaFormat and renderJavadoc pass for the grouping module.

Not recommended: change the managers to match the bullet

Making the managers' topNGroups include the offset would break GroupingSearch's documented groupLimit, the TestGrouping oracle and every existing manager caller, and reverse the choice made in the #15936 review. Deleting only the "7 groups" sentence would leave "how many top groups to keep" as ambiguous as the managers' @param texts.

Trade-offs

Not run.

Downstream note

Applications that page grouped results through the two collector managers with the rule from this text show oversized, overlapping pages; those that use GroupingSearch are not affected.

How it was found

Non-formal. While modelling the grouping module's offset + limit sums, which treats groupOffset and topNGroups as separate inputs as the managers do, the finder noticed that the package text counts the offset inside topNGroups. A probe and a public-API JUnit on the stock jar confirmed it; verifiers, cross-model reviews and an adjudication followed.

The method in full87 words
  • First signal: reading the package text against a hunt unit's site model (int offset + limit sums in grouping), 2026-10-28. No model produced or witnessed the defect, so the finding is non-formal.
  • Repro: the finder's JUnit (15 one-document groups, both managers and a plain-collector control) failed 2 of 3 on main with assertions on and off and on the 10.6.0 release commit; the control passed.
  • Minimal repro (2026-10-29): cut to one method, the first-pass manager only, 13 groups; fails on main 645d4fe14a20 with assertions on and off.

Prior reports, history and checks

Prior reports

No prior report found (2026-10-28, refreshed 2026-11-01): GitHub issues and PRs incl. the JIRA archive, commits, mailing lists, downstream projects, the web, and all 414 open PRs' changed files. Nearest: #15936, where the managers' page-size meaning was chosen, and #16385, the rewrite that made the contradiction direct.

Searches5 searches

The exact phrase "you'll get 7 groups" has no hits on GitHub or in the JIRA archive. GitHub search does not index diff text, and private channels cannot be searched.

What was searched, when, and what turned up
WhereWhenHowFound
GitHub issues, PRs and comments (apache/lucene incl. the migrated JIRA archive), commits, downstream repositories, dev@/java-user@/issues@/commits@, the web2026-10-28three scripted prior-report searches: 59 queries (225 distinct hits), 51 queries (216 hits), and 40 queries plus 20 manual GitHub searchesno report; #15936 is context only
GitHub manual (apache/lucene, apache/solr, OpenSearch, Elasticsearch) and JIRA text search2026-10-28"you'll get 7 groups", "7 groups back", topNGroups, groupOffset, "grouping package-info", GroupingSearch offset0 hits for the phrases; topNGroups in JIRA only LUCENE-3778
Review threads and issue comments2026-10-28#15936, #16385, #16508, #16292, #15574 and all 23 comments of LUCENE-1421 readno comment mentions the bullet
Open PRs2026-10-28the changed-file lists of all 414 open apache/lucene PRsnone touches the grouping package-info.java
Refresh2026-11-01GitHub search and API: issues and PRs updated since 2026-10-28; commits on the filenothing about grouping offsets; last commit on the file still #16292 (2026-10-06)

History

The sentence dates from LUCENE-1421 (3.2, 2011) and has disagreed with the page's own examples since. The managers shipped in 10.5.0 with page-size semantics; #16385 (10.6.0) made the page name only them, a direct contradiction. Not fixed upstream.

Versions tested5 rows

Direct contradiction introduced by a6e670d6ed1 "Remove unused public methods in AllGroupHeadsCollector (#16385)" (gaobinlong, reviewed by javanna, who asked for the package-info update; 2026-08-12, milestone 10.6.0; backport d729cbfd11f). It renamed the classes in the two-pass paragraph and turned the Block example into the manager form, and left the inputs list unchanged.

Older ingredients: the bullet and a two-pass example with groupOffset+topNGroups from LUCENE-1421 (9083cedb922, ee2154e4a39, 2011-05-14, 3.2); the Block example with groupOffset+topNGroups from LUCENE-3129 (375c1abdbf4 on trunk, 29ac743e238 on branch_3x, 2011-06-02; first released in 3.3, through the branch_3x commit); the managers from #15574 (636e8e9f587) and #15936 (d27d080d101), June 2026, first released in 10.5.0.

Not fixed upstream (a snapshot): upstream main f815e679bf6c (2026-11-01) still has the sentence; the last commit to the file is d148795b182 (#16292, 2026-10-06), on other lines. The parent of a6e670d6ed1 was not built: the commit changes no code the repro runs.

Versions and branches executed, with results
Version or commitVerdictResult
main 645d4fe14a20 affectedminimal test fails with assertions on and off (gradle and standalone); the page names the managers and no plain collector
10.6.0 (release jars; release commit 4ae946e5a9b) affectedminimal test fails with assertions on and off; the hunt's test fails 2 of 3
10.5.0, 10.5.1, 10.5.2 page named the plain collectorthe managers already return 10 (the hunt's test on all three, the minimal test on 10.5.0), but the page named FirstPassGroupingCollector, for which the sentence holds (control returns 7); its Block example already returned 10
9.12.3, 10.4.0 examples onlyno managers; the page's Block example returns 10 against the bullet's 7
3.2, 3.3, 9.0.0 text readthe same bullet next to examples that pass groupOffset+topNGroups (not run)

Independent checks

The finder, an independent reproducer, a works-as-designed cross-model review (OpenAI GPT-5.6), a devil's-advocate verifier, the adjudication and a credibility audit all call it a real documentation defect; a reading-only refute review agreed on substance. This doc's gradle runs and release-jar runs confirm it on main, 10.6.0 and 10.5.0.

Checks and verdicts279 words

The cross-model reviews could only read the code (they could not run Java); no executing cross-model review was done. The adjudication required three corrections, applied here: the mismatch is long-standing, the managers behave this way since 10.5.0, and the fix names both plain collectors.

Each independent check and its verdict
CheckVerdictWhat it found
finder fails 2 of 3Both managers return 10 for (3, 10) on main and the 10.6.0 release commit, assertions on and off; the plain-collector control returns 7.
cross-model refute review not real (procedural), agrees on substanceOpenAI GPT-5.6, reading only: could not run code, so "not reproduced"; agreed with the root cause and called it a genuine documentation defect. A separate re-run failed 2 of 3.
Independent reproducer real (doc)Own 6-test JUnit: 3 fail on main and 10.5.2, 3 controls pass; overlapping pages; the old Block example returns 10 on 9.12.3, 10.5.2 and main; corrected the version story.
Cross-model works-as-designed lens real, highOpenAI GPT-5.6, reading only: the intended-behaviour case covers the code, not the text; the fix should also name BlockGroupingCollector.
Devil's advocate real (doc, minor)Built the strongest intended-behaviour case and showed why it fails; re-runs fail on main and 10.5.2.
adjudication confirmed, highRe-ran the independent reproduction's test on main with assertions on and off (3 of 6 fail, controls pass); required the three corrections.
credibility audit reproducesRe-ran the finder's test with a fresh seed: 2 of 3 fail as expected.
this doc fails; fix A validatedGradle before and after fix A on 645d4fe14a20; minimal test on the 10.6.0 and 10.5.0 release jars, assertions on and off.

Diagram file

The diagram as a still image, generated with AI and checked against the code. It is under 10 MB, the limit for a GitHub attachment.

Sources and provenance

Finding record, Lucene and upstream sources, repro files and diagram notes1,422 words

Finding record

LFV-961, snapshot of 2026-11-06

Registry title
The grouping package documentation says groupOffset 3 with topNGroups 10 returns 7 groups, but FirstPassGroupingCollectorManager and BlockGroupingCollectorManager, which the page names and shows since #16385 (10.6.0), return 10: for them topNGroups is the page size after the offset (documentation defect; the inputs list has also disagreed with the page's own example code since 3.2)
Area
grouping / package documentation (package-info.java inputs list vs the collector managers it names)
Priority
Low, rank 1235: This grouping doc defect, mostly new in 10.6.0, can silently lead users who follow the package prose into oversized, overlapping pages.
Impact class
minor
Impact note
Documentation only, in the optional grouping module: a developer who writes paging code from the package overview's inputs list and passes topNGroups = groupOffset + pageSize to FirstPassGroupingCollectorManager or BlockGroupingCollectorManager (the classes the 10.6 page names) gets pages that are larger than the page size and overlap the next page (page size 5 on 15 groups: the page at groupOffset 5 returns g05..g14 and the next page g10..g14); every returned group still matches the query, the code is correct and intended, GroupingSearch's own javadoc is right, and no scale or non-default setting is needed; the direct contradiction is in main and 10.6.0 (released 2026-10-29) only, while the inputs list has disagreed with the page's own example code in every release since 3.2 and the managers have behaved this way since 10.5.0.
Summary
You read the grouping package overview, which says topNGroups is 'how many top groups to keep' and that groupOffset 3 'means you'll get 7 groups back (assuming topNGroups is 10)', and which names FirstPassGroupingCollectorManager as the first pass and builds BlockGroupingCollectorManager(groupSort, groupOffset, topNGroups, ...), so you expect groupOffset 3 with topNGroups 10 to return 7 groups. Instead both managers, like GroupingSearch, treat topNGroups as the number of groups to return after the offset and return 10 groups (ranks 4 to 13). The '7 groups' sentence is right only for the plain FirstPassGroupingCollector and BlockGroupingCollector, which the page named until #16385 rewrote it for the managers, so paging code written from the sentence gets oversized, overlapping pages.
Introduced
a6e670d6ed1b4342accf75aed0d63ea11fde6645 (#16385 'Remove unused public methods in AllGroupHeadsCollector', by gaobinlong, reviewed and approved by javanna, who asked for the package-info update; milestone 10.6.0; backport d729cbfd11f in branch_10x and branch_10_6) [read: the commit rewrote the two-pass paragraph to name FirstPassGroupingCollectorManager and TopGroupsCollectorManager instead of the collectors, and the single-pass example from new BlockGroupingCollector(groupSort, groupOffset+topNGroups, ...) to new BlockGroupingCollectorManager(groupSort, groupOffset, topNGroups, ...), and left the inputs list unchanged; exec at the version boundary: on 10.5.2 the page names FirstPassGroupingCollector, for which the sentence holds (control returns 7), and on the branch_10_6 head 4ae946e5a9b (10.6.0, released 2026-10-29) the named managers return 10; the commit's parent was not built, because the commit changes no code that the repro runs]. Older ingredients [read: git log -S, blame]: the inputs-list sentence and a two-pass example that already passed groupOffset+topNGroups come from LUCENE-1421 (9083cedb922 on trunk, ee2154e4a39 on branch_3x, by mikemccand, 2011-05-14; first released in 3.2); the BlockGroupingCollector example with groupOffset+topNGroups from LUCENE-3129 (375c1abdbf4, 2011-06-02; 3.3); the managers' page-size semantics from #15574 (636e8e9f587, 2026-06-03) and #15936 (d27d080d101, 2026-06-15, where the author chose groupOffset+topNGroups in review to match GroupingSearch), both first released in 10.5.0. (2026-08-12 (author date of a6e670d6ed1, 21:16:52 +0800 = 13:16:52Z, merged the same minute; backport d729cbfd11f committed 2026-08-12 15:27:39 +0200); older ingredients 2011-05-14 (LUCENE-1421) and 2011-06-02 (LUCENE-3129))
First affected version
10.6.0, released 2026-10-29 (branch_10_6 head 4ae946e5a9b, tag releases/lucene/10.6.0) for the direct contradiction with the classes the page names [exec + read]. Not in this form in 10.5.0-10.5.2, whose page names the collectors, although the managers already return 10 there [exec]. The weaker form, the inputs list disagreeing with the page's own example code (which passed groupOffset+topNGroups and so returns 10 groups for topNGroups 10), is in every release since 3.2 [exec with the pre-10.6 Block example on 9.12.3, 10.4.0 and 10.5.2; text read on 3.2, 3.3, 9.0.0, 9.12.3, 10.4.0, 10.5.0 and 10.5.2].
Affected versions
Direct contradiction (the page names the managers): main a9779ef3c193 [exec: the finder's JUnit fails 2 of 3 with -ea and -da, seed DEADBEEF, the collector control passes; the independent reproduction's JUnit fails 3 of 6 with -ea and -da, seeds C0FFEE and the adjudication's 5EED1234, its 3 controls pass; the intended-behaviour check re-ran the finder's JUnit with -ea and -da; re-run -da, seed 5EED3870, 2 of 3], the branch_10_6 head 4ae946e5a9b (10.6.0, released 2026-10-29) [exec, -ea and -da], branch_10x (10.7.0-SNAPSHOT, contains d729cbfd11f) [read]; upstream main 645d4fe14a20 (2026-10-28) still has the sentence [read: the last commit to the file is d148795b182, #16292, 2026-10-06, other lines]. Long-standing mismatch between the inputs list and the page's own example code: 3.2 to 10.5.2 and still on 10.6/main [exec: the independent reproduction's pre-10.6 BlockGroupingCollector example returns 10, not 7, on 9.12.3, 10.4.0, 10.5.2, 10.6.0 (branch_10_6 head) and main; read on 3.2, 3.3 and 9.0.0]. The managers' behaviour, unchanged since they shipped: 10.5.0, 10.5.1 and 10.5.2 return 10 for (3, 10) [exec]; 9.12.3 and 10.4.0 have no managers (the JUnit does not compile) [exec]. All runs JDK 25.0.3 on Linux aarch64.
Status
documented (brief, checks and a per-finding doc exist)
Verification
Confirmed by AI-run checks: an independent repro, a cross-model review that disputed it (the adjudication ruled it real), an intended-behaviour check, a prior-report search and a skeptical audit; not yet triaged by a person.
Found by
non-formal: code reading
Upstream and prior reports
none found (searched 2026-10-28): three scripted prior-report searches (the follow-up's 59 queries over GitHub issues and PRs, commits, downstream repositories, dev@/java-user@/issues@/commits@ lists, the JIRA archive and the web, 225 distinct hits; the finder's 51 queries, 216 hits; the intended-behaviour check's 40 queries plus 20 manual GitHub searches), manual GitHub searches over apache/lucene, apache/solr, OpenSearch and Elasticsearch (exact phrase "you'll get 7 groups": 0), JIRA text searches ('"7 groups back"': 0; topNGroups: only LUCENE-3778), all 23 comments of LUCENE-1421, the review threads of #15936, #16385, #16508, #16292 and #15574, and the changed-file lists of all 414 open apache/lucene PRs (none touches grouping package-info.java). Related, not reports: #15936 (the review thread where the managers' page-size semantics were chosen on purpose), #16385 (the rewrite that made the contradiction direct; no review comment mentions the bullet), #16508 and #16292 (other sentences of the same page), #4851 (LUCENE-3778, GroupingSearch's offset/limit), #15995 (a different withinGroupOffset code bug). Not fixed upstream: upstream main 645d4fe14a20 (2026-10-28) still has the sentence. Refresh 2026-10-28 (GitHub search and REST API): none of the 72 apache/lucene PRs updated since 2026-10-26 touches lucene/grouping; 'topNGroups groupOffset' finds only #4851; '"7 groups back"' finds nothing; no new commit on the file.
Lucene commit tested
main 645d4fe14a20: the minimal test through gradle (2026-11-01) and standalone (2026-10-29), assertions on and off; fix A in a scratch worktree (format check, javadoc build, the whole grouping test module); the minimal test on the 10.6.0 and 10.5.0 release jars, assertions on and off (2026-11-01). The hunt's and verifiers' tests ran on main a9779ef3c193 (same grouping sources), the 10.6.0 release commit 4ae946e5a9b and 10.5.0 to 10.5.2; the page's pre-10.6 Block example on 9.12.3, 10.4.0, 10.5.2, 10.6.0 and main.

Lucene and upstream sources

Finding record
  • a Lucene main checkout lucene/grouping/src/java/org/apache/lucene/search/grouping/package-info.java:37-41 (inputs list), :49-51 (first pass = FirstPassGroupingCollectorManager), :129-137 (BlockGroupingCollectorManager example); FirstPassGroupingCollectorManager.java:66-67 and :82-83 (@param), :106-111 (groupOffset + topNGroups), :125 (SearchGroup.merge(allGroups, groupOffset, topNGroups, groupSort)); BlockGroupingCollectorManager.java:87-88 (@param), :141-143, :158-159 (mergeBlockGroups with groupOffset, topNGroups); GroupingSearch.java:123 (@param groupLimit); FirstPassGroupingCollector.java:72, :86 and BlockGroupingCollector.java:204 ('How many top groups to keep') [read by the registration check]
  • a Lucene clone releases/lucene/10.5.2 package-info.java:40, :50, :128 (names FirstPassGroupingCollector; Block example with groupOffset+topNGroups); in the same clone, d729cbfd11f is an ancestor of 4ae946e5a9b [read by the registration check]
The finding in 30 seconds
  • lucene/grouping/src/java/org/apache/lucene/search/grouping/package-info.java:37-41, :49-52, :129-137 (at 645d4fe14a20)
  • lucene/grouping/src/java/org/apache/lucene/search/grouping/FirstPassGroupingCollectorManager.java:66-67, :82-83, :106-111, :125 (at 645d4fe14a20)
  • lucene/grouping/src/java/org/apache/lucene/search/grouping/SearchGroup.java:278-311 (at 645d4fe14a20)
  • lucene/grouping/src/java/org/apache/lucene/search/grouping/BlockGroupingCollectorManager.java:87-88, :141-144, :158-159 (at 645d4fe14a20)
  • lucene/grouping/src/java/org/apache/lucene/search/grouping/GroupingSearch.java:122-123, :148 (at 645d4fe14a20)
  • lucene/grouping/src/java/org/apache/lucene/search/grouping/FirstPassGroupingCollector.java:72, :86, :133-136; BlockGroupingCollector.java:204, :256-261 (at 645d4fe14a20)
Who is affected
  • lucene/grouping/src/java/org/apache/lucene/search/grouping/FirstPassGroupingCollectorManager.java:50, :106-111; BlockGroupingCollectorManager.java:68 (at 645d4fe14a20)
Why it looks unintended
  • lucene/grouping/src/test/org/apache/lucene/search/grouping/TestGrouping.java:676, :680, :1110-1112, :1300-1309 (at 645d4fe14a20)
  • lucene/grouping/src/java/org/apache/lucene/search/grouping/GroupingSearch.java:123 (at 645d4fe14a20)
  • lucene/grouping/src/test/org/apache/lucene/search/grouping/TestGrouping.java:173-174; TestGroupingSearch.java:125-126 (hit counts; at 645d4fe14a20)
Suggested fix
  • lucene/grouping/src/java/org/apache/lucene/search/grouping/TopGroupsCollectorManager.java:50-56 (withinGroupOffset only), AllGroupHeadsCollectorManager.java, TermGroupFacetCollectorManager.java (no groupOffset or topNGroups; at 645d4fe14a20)
History
  • a Lucene clone tag releases/lucene/10.6.0 = 4ae946e5a9b, contains d729cbfd11f
Diagrams
  • lucene/grouping/src/java/org/apache/lucene/search/grouping/FirstPassGroupingCollectorManager.java:110, :125; BlockGroupingCollectorManager.java:143, :159 (at 645d4fe14a20)

Other statements trace to the hunt's working files (models, logs, reviews and briefs), which are not published.

Repro files

About the diagrams

Built 2026-11-06 from the hunt's record of this finding and its doc file. Lucene main under test in this hunt: ea57360912ed. Every listed finding was re-run on Lucene main e1faeba08bde on 2026-11-02.