Both managers give each collector a pool of groupOffset + topNGroups groups, then merge, skip groupOffset groups and return up to topNGroups. So topNGroups is the page size. The inputs list describes the plain collectors, whose topNGroups includes the offset; since 10.6.0 the page names the managers and no plain collector, and their own @param texts do not correct it.
- A grouped search ranks groups, for example all books by one author, by
groupSort. To show page 2 at 5 groups per page you skip 5 groups (groupOffset) and ask for the next 5. - The plain first-pass collector
FirstPassGroupingCollector(selector, sort, topNGroups)keepstopNGroupsgroups in total, andgetTopGroups(groupOffset)drops the firstgroupOffsetof them: offset 3 of 10 leaves 7. - Since 10.5.0 the module also has collector managers for concurrent search,
FirstPassGroupingCollectorManager(selector, sort, groupOffset, topNGroups)andBlockGroupingCollectorManager(sort, groupOffset, topNGroups, ...). Since 10.6.0 the package overview names these and no plain collector.
groupOffset 3 and topNGroups 10: the page promises 7 groups, the two managers return 10 (g03 to g12), and only the plain collectors return 7. Select the still to open it at full size. Generated with AI, checked against the code.- You page grouped results with the first-pass class the grouping overview names,
FirstPassGroupingCollectorManager. Its inputs list saystopNGroupsis "how many top groups to keep" and thatgroupOffset3 "means you'll get 7 groups back (assumingtopNGroupsis 10)". - On an index with 13 groups you pass
groupOffset3 andtopNGroups10, and you expect 7 groups: ranks 4 to 10. - You get 10 groups: ranks 4 to 13.
BlockGroupingCollectorManagerandGroupingSearch.search(searcher, query, 3, 10)also return 10; only the plain collectors return 7:FirstPassGroupingCollectorwithgetTopGroups(3), andBlockGroupingCollectorwithtopNGroups10 andgroupOffset3. - Paging written from the sentence, 5 groups per page (
groupOffset5p,topNGroups5p+5), over 15 groups g00 to g14: page 1 returns g05 to g14 (10 groups) and page 2 repeats g10 to g14. - Why: each manager keeps
groupOffset + topNGroupsgroups per collector, then skipsgroupOffsetand returns up totopNGroups, asGroupingSearchdocuments. The sentence describes the plain collectors, which the page stopped naming in 10.6.0 (#16385).
Developers who page grouped results with FirstPassGroupingCollectorManager or BlockGroupingCollectorManager (experimental, since 10.5.0) and take topNGroups from the package overview, its most explicit definition (the managers' @param texts read the same way). Their pages are larger than the page size and repeat groups of the next page; every group still matches the query. GroupingSearch users are not affected.
Mechanism, terms and numbers566 words
The lucene-grouping package overview defines the two paging inputs of a grouped search: topNGroups is "how many top groups to keep. For example, 10 means the top 10 groups are computed", and groupOffset 3 "means you'll get 7 groups back (assuming topNGroups is 10). This is useful for paging, where you might show 5 groups per page". Since #16385 (10.6.0) the same page names FirstPassGroupingCollectorManager as the first pass and builds BlockGroupingCollectorManager(groupSort, groupOffset, topNGroups, ...). Both managers, like GroupingSearch, treat topNGroups as the number of groups to return after the offset, so (3, 10) returns 10 groups, ranks 4 to 13. The "7 groups" rule holds only for the plain FirstPassGroupingCollector and BlockGroupingCollector, whose topNGroups includes the offset, and the 10.6 page names neither. The code is correct and intended; the text needs fixing. The bullet has also disagreed with the page's own example code since 3.2.
:37-41: "topNGroups: how many top groups to keep. For example, 10 means the top 10 groups are computed." / "groupOffset: which "slice" of top groups you want to retrieve. For example, 3 means you'll get 7 groups back (assumingtopNGroupsis 10). This is useful for paging, where you might show 5 groups per page.":49-52: "the first pass (FirstPassGroupingCollectorManager) gathers the top groups, and the second pass (TopGroupsCollectorManager) gathers documents within those groups".:129-137:new BlockGroupingCollectorManager<>(groupSort, groupOffset, topNGroups, needsScores, lastDocPerGroup, withinGroupSort, docOffset, docsPerGroup). No plain collector is named anywhere on the page.
// FirstPassGroupingCollectorManager.java:106-125
public FirstPassGroupingCollector<T> newCollector() throws IOException {
return new FirstPassGroupingCollector<>(
groupSelectorFactory.get(), groupSort, groupOffset + topNGroups, ignoreDocsWithoutGroupField);
}
... reduce(...): allGroups.add(collector.getTopGroups(0)) for each collector, then
return SearchGroup.merge(allGroups, groupOffset, topNGroups, groupSort);SearchGroup.mergekeeps a queue ofoffset + topN(SearchGroup.java:281), skips the firstoffsetgroups (if (count++ >= offset),:304) and stops when it holdstopN(:309-310).BlockGroupingCollectorManagerdoes the same:new BlockGroupingCollector(groupSort, groupOffset + topNGroups, ...)(BlockGroupingCollectorManager.java:141-144), thenTopGroups.mergeBlockGroups(shardGroupsList, groupSort, groupOffset, topNGroups, withinGroupSort)(:158-159).GroupingSearchdocuments this meaning: "@param groupLimitThe number of groups to return from the specified group offset" (GroupingSearch.java:123), and passesgroupOffset, groupLimitto the first-pass manager (:148).- Where "7" is right: the plain collectors take a pool size that includes the offset, "
@param topNGroupsHow many top groups to keep" (FirstPassGroupingCollector.java:72,:86;BlockGroupingCollector.java:204), and the offset goes togetTopGroups(FirstPassGroupingCollector.java:136;BlockGroupingCollector.java:260-261). - The managers' own javadoc reads like the bullet: "
@param groupOffsetThe offset in the collected groups" and "@param topNGroupsthe number of top groups to collect" (FirstPassGroupingCollectorManager.java:66-67,:82-83; the first is word for wordFirstPassGroupingCollector.getTopGroups's@param,:133), and "the number of top groups to collect" (BlockGroupingCollectorManager.java:88). OnlyGroupingSearchis unambiguous.
13 one-document groups g0 to g12 by index order, (groupOffset 3, topNGroups 10): FirstPassGroupingCollectorManager returns 10 groups, g3 to g12 (the minimal test; the first group checked in this doc's fixed-text run). 15 groups g00 to g14 (the independent reproduction's test): the manager returns g03 to g12, BlockGroupingCollectorManager 10 groups, GroupingSearch.search(searcher, q, 3, 10) 10 groups from g03, and the plain FirstPassGroupingCollector(selector, sort, 10).getTopGroups(3) 7. Paging from the text, groupOffset 5p and topNGroups 5p+5: page0=[g00..g04], page1=[g05..g14], page2=[g10..g14].
Exposure and likelihood205 words
- Who: code written against the two collector managers from the overview's inputs list. Users of
GroupingSearch, and anyone who passes the page size astopNGroups(the meaningGroupingSearchdocuments for itsgroupLimit), get the intended page size. - Default, not a special setting: no index size, codec or flag is involved. The minimal repro uses 13 one-document groups, and 11 already show the mismatch: with
(3, 10)the first-pass manager returns 6, 7, 8, 9 and 10 groups for 9, 10, 11, 12 and 13 groups, so from 11 groups on it returns more than the sentence's 7. Since 10.6.0 the page names the managers and no plain collector, so the overview now leads its readers to them. - Effect: silent. No exception; pages look plausible. Nothing excluded or deleted appears: every group comes from the same query, filters and reader. The first pass also does somewhat more work than meant (each collector keeps about
2 x groupOffset + pageSizegroups instead ofgroupOffset + pageSize). - Likelihood: plausible but limited. The managers are new (10.5.0) and
@lucene.experimental, most users go throughGroupingSearch, and no one has reported the mismatch in the roughly 15 years the bullet has disagreed with the page's examples.
One JUnit test, public API only: 13 one-document groups, a default IndexWriterConfig, ByteBuffersDirectory, and FirstPassGroupingCollectorManager with the overview's own numbers, groupOffset 3 and topNGroups 10. It asserts the documented 7 groups. Deterministic, single thread, the same result with assertions on and off.
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.lucene.search.grouping;
import java.util.Collection;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.SortedDocValuesField;
import org.apache.lucene.index.DirectoryReader;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.MatchAllDocsQuery;
import org.apache.lucene.search.Sort;
import org.apache.lucene.store.ByteBuffersDirectory;
import org.apache.lucene.store.Directory;
import org.apache.lucene.tests.util.LuceneTestCase;
import org.apache.lucene.util.BytesRef;
public class TestGroupingPackageDocOffset extends LuceneTestCase {
public void testOffsetThreeOfTopTenGivesSevenGroups() throws Exception {
try (Directory dir = new ByteBuffersDirectory()) {
try (IndexWriter w = new IndexWriter(dir, new IndexWriterConfig())) {
for (int g = 0; g < 13; g++) {
Document doc = new Document();
doc.add(new SortedDocValuesField("g", new BytesRef("g" + g)));
w.addDocument(doc);
}
}
try (DirectoryReader r = DirectoryReader.open(dir)) {
Collection<SearchGroup<BytesRef>> groups =
new IndexSearcher(r)
.search(
MatchAllDocsQuery.INSTANCE,
new FirstPassGroupingCollectorManager<>(
() -> new TermGroupSelector("g"), Sort.INDEXORDER, 3, 10));
// package-info: groupOffset 3 "means you'll get 7 groups back (assuming topNGroups is 10)"; actual 10
assertEquals(7, groups.size());
}
}
}
}
cp $REPRO/TestGroupingPackageDocOffset.java $LUCENE/lucene/grouping/src/test/org/apache/lucene/search/grouping/
cd $LUCENE
./gradlew :lucene:grouping:test --tests org.apache.lucene.search.grouping.TestGroupingPackageDocOffset -Ptests.seed=DEADBEEF -Ptests.asserts=true
# assertions off: -Ptests.asserts=false (same failure)
- Expected
- 7 groups, as the package overview promises for
groupOffset3 andtopNGroups10. - Observed
AssertionError: expected:<7> but was:<10>: the manager returns the groups ranked 4 to 13.
More files, environment, full output224 words
- Smallest input. 13 groups are the fewest for which the manager returns its full
topNGroups(10); 11 or 12 would also differ from 7 but not show the full 10. - Releases. The same test fails the same way on the 10.6.0 and 10.5.0 release jars, assertions on and off. It needs the managers, so it does not apply before 10.5.0, and on 10.5.x the page did not name them yet.
- Not a regression test. It asserts the documented promise, so it is evidence for a doc fix: with the fix it would be dropped, or kept with its one assertion changed to the corrected text's example (10 groups, the first being
g3), which this doc ran.
| lucene | main 645d4fe14a20 (11.0.0-SNAPSHOT); also fails on the 10.6.0 and 10.5.0 release jars |
|---|---|
| jdk | OpenJDK 25.0.3 |
| os | Linux aarch64 |
cp $REPRO/TestGroupingPackageDocOffset.java $LUCENE/lucene/grouping/src/test/org/apache/lucene/search/grouping/
cd $LUCENE
./gradlew :lucene:grouping:test --tests org.apache.lucene.search.grouping.TestGroupingPackageDocOffset -Ptests.seed=DEADBEEF -Ptests.asserts=true
# assertions off: -Ptests.asserts=false (same failure)
# gradle, main 645d4fe14a20, seed DEADBEEF, -Ptests.asserts=true
TestGroupingPackageDocOffset > testOffsetThreeOfTopTenGivesSevenGroups FAILED
java.lang.AssertionError: expected:<7> but was:<10>
at org.apache.lucene.search.grouping.TestGroupingPackageDocOffset.testOffsetThreeOfTopTenGivesSevenGroups(TestGroupingPackageDocOffset.java:52)
BUILD FAILED
# -Ptests.asserts=false: the same AssertionError
# release jars 10.6.0 and 10.5.0, assertions on and off: 1 run, 1 failed, expected:<7> but was:<10>
# the same test with its assertion changed to the fixed text (10 groups, first g3), fix A applied: passes, assertions on and off- The page contradicts itself: its own examples always passed
groupOffset + topNGroupsto the plain collectors, which returns 10 groups, not 7. - The managers' meaning was chosen in the #15936 review to match
GroupingSearch, andTestGroupingchecks it. - #16385 renamed the classes on the page but kept the inputs list; nothing endorses it for the managers.
The full argument158 words
- Self-contradiction since 2011: LUCENE-1421's two-pass example passed
groupOffset+topNGroupstoFirstPassGroupingCollector, and LUCENE-3129's Block examplenew BlockGroupingCollector(groupSort, groupOffset+topNGroups, ...)withgetTopGroups(withinGroupSort, groupOffset, ...). Run withtopNGroups10 andgroupOffset3, that example returns 10 groups on 9.12.3, 10.4.0, 10.5.2, 10.6.0 and main. - The code is intended: in the #15936 review a reviewer asked whether the Block manager should "also use groupOffset", and the author changed it to
groupOffset+topNGroupsto matchGroupingSearch.TestGrouping's oracle computeslimit = Math.min(groupOffset + topNGroups, groups.size())and returnslimit - groupOffsetgroups (TestGrouping.java:676,:680), and passesgroupOffset, topNGroupsstraight to both managers (:1110-1112,:1300-1309). - Nothing endorses the bullet for the managers: #16385 rewrote the class names and the example at a reviewer's request and left the inputs list; no review comment discusses it, no test uses
(3, 10)or asserts the bullet's 7 groups (theassertEquals(7, ...)calls inTestGrouping.java:173-174andTestGroupingSearch.java:125-126check hit counts).
The devil's-advocate check139 words
Strongest case (a separate intended-behaviour verifier and a cross-model reviewer, OpenAI GPT-5.6, works-as-designed lens): the inputs list is a 2011 conceptual description of the low-level collectors; "how many top groups to keep" is their @param word for word, and for them "7 groups" is exactly right; the managers' meaning was decided in review and is tested; GroupingSearch documents it; the looseness is 15 years old and unreported.
Why it fails: all of that shows the code is right, which is the claim. The 10.6 page names no plain collector; it names the managers and shows a manager example, and for every paging API it names (both managers and GroupingSearch) its example "3 means you'll get 7 groups back" is false. The managers' own @param texts read like the bullet, so a manager user has nothing that corrects it.
Recommended A. Rewrite the topNGroups and groupOffset bullets, naming both plain collectors (recommended)
Rewrite the two bullets in the package overview: topNGroups is how many groups to return counted from groupOffset, so 3 with 10 returns the groups ranked 4 to 13, as for GroupingSearch and the managers; add that the plain FirstPassGroupingCollector and BlockGroupingCollector take a count that includes the offset. Text only; the javadoc build and grouping tests pass.
* <li><code>topNGroups</code>: how many groups to return, counted from <code>groupOffset</code>.
* For example, 10 means up to 10 groups are returned.
* <li><code>groupOffset</code>: which "slice" of top groups you want to retrieve. For example, 3
* with <code>topNGroups</code> 10 returns the groups ranked 4 to 13. This is useful for
* paging, where you might show 5 groups per page: page <code>p</code> (counting from 0) uses
* <code>groupOffset</code> 5*p and <code>topNGroups</code> 5. This is how {@link
* org.apache.lucene.search.grouping.GroupingSearch} and the collector managers below read
* these inputs. The low-level collectors ({@link
* org.apache.lucene.search.grouping.FirstPassGroupingCollector}, {@link
* org.apache.lucene.search.grouping.BlockGroupingCollector}) instead keep a number of groups
* that includes the offset: create them with <code>groupOffset + topNGroups</code> and pass
* <code>groupOffset</code> to <code>getTopGroups</code>.All options and trade-offs3 options ยท 485 words
The behaviour is intended, so every option is a text change. Option A was applied in a scratch worktree of 645d4fe14a20; option B was built on top of it.
A. Rewrite the topNGroups and groupOffset bullets, naming both plain collectors (recommended) (validated by the checks)
Replace package-info.java:37-41 with the text below. A shorter variant keeps only the first two bullets plus one sentence on the plain collectors.
* <li><code>topNGroups</code>: how many groups to return, counted from <code>groupOffset</code>.
* For example, 10 means up to 10 groups are returned.
* <li><code>groupOffset</code>: which "slice" of top groups you want to retrieve. For example, 3
* with <code>topNGroups</code> 10 returns the groups ranked 4 to 13. This is useful for
* paging, where you might show 5 groups per page: page <code>p</code> (counting from 0) uses
* <code>groupOffset</code> 5*p and <code>topNGroups</code> 5. This is how {@link
* org.apache.lucene.search.grouping.GroupingSearch} and the collector managers below read
* these inputs. The low-level collectors ({@link
* org.apache.lucene.search.grouping.FirstPassGroupingCollector}, {@link
* org.apache.lucene.search.grouping.BlockGroupingCollector}) instead keep a number of groups
* that includes the offset: create them with <code>groupOffset + topNGroups</code> and pass
* <code>groupOffset</code> to <code>getTopGroups</code>.Trade-offs
Text only. Keeps the page's paging example and makes it concrete; names both plain collectors, as the adjudication and the cross-model works-as-designed review asked (the hunt's first diff named only the first-pass one). "The collector managers below" means the two that take these inputs (FirstPassGroupingCollectorManager, BlockGroupingCollectorManager); the other managers on the page (TopGroupsCollectorManager, AllGroupHeadsCollectorManager, TermGroupFacetCollectorManager) take neither.
What the checks validated
:lucene:grouping:checkGoogleJavaFormatand:lucene:grouping:renderJavadocpass; the rendered package summary shows the new text with working links toGroupingSearch,FirstPassGroupingCollectorandBlockGroupingCollector.- The minimal test fails before the fix with assertions on and off; the same test with its one assertion changed to the new text's example (10 groups, first
g3, ranks 4 to 13) passes with assertions on and off. - The whole
:lucene:grouping:testwith assertions on: 15 classes, 57 tests, 0 failures (TestGrouping7/7);TestGroupingagain with assertions off: 0 failures. SeedDEADBEEF, OpenJDK 25.0.3.
FirstPassGroupingCollectorManager (both constructors): @param groupOffset "how many top groups to skip", @param topNGroups "how many groups to return after the skipped ones"; BlockGroupingCollectorManager: @param topNGroups "how many groups to return, starting at groupOffset".
Trade-offs
Removes the ambiguity at the source a manager user reads first; five @param lines reworded in three constructor javadocs (no lines added).
What the checks validated
With A applied: checkGoogleJavaFormat and renderJavadoc pass for the grouping module.
Making the managers' topNGroups include the offset would break GroupingSearch's documented groupLimit, the TestGrouping oracle and every existing manager caller, and reverse the choice made in the #15936 review. Deleting only the "7 groups" sentence would leave "how many top groups to keep" as ambiguous as the managers' @param texts.
Trade-offs
Not run.
Applications that page grouped results through the two collector managers with the rule from this text show oversized, overlapping pages; those that use GroupingSearch are not affected.
Non-formal. While modelling the grouping module's offset + limit sums, which treats groupOffset and topNGroups as separate inputs as the managers do, the finder noticed that the package text counts the offset inside topNGroups. A probe and a public-API JUnit on the stock jar confirmed it; verifiers, cross-model reviews and an adjudication followed.
The method in full87 words
- First signal: reading the package text against a hunt unit's site model (int offset + limit sums in grouping), 2026-10-28. No model produced or witnessed the defect, so the finding is non-formal.
- Repro: the finder's JUnit (15 one-document groups, both managers and a plain-collector control) failed 2 of 3 on main with assertions on and off and on the 10.6.0 release commit; the control passed.
- Minimal repro (2026-10-29): cut to one method, the first-pass manager only, 13 groups; fails on main
645d4fe14a20with assertions on and off.
No prior report found (2026-10-28, refreshed 2026-11-01): GitHub issues and PRs incl. the JIRA archive, commits, mailing lists, downstream projects, the web, and all 414 open PRs' changed files. Nearest: #15936, where the managers' page-size meaning was chosen, and #16385, the rewrite that made the contradiction direct.
Searches5 searches
The exact phrase "you'll get 7 groups" has no hits on GitHub or in the JIRA archive. GitHub search does not index diff text, and private channels cannot be searched.
| Where | When | How | Found |
|---|---|---|---|
| GitHub issues, PRs and comments (apache/lucene incl. the migrated JIRA archive), commits, downstream repositories, dev@/java-user@/issues@/commits@, the web | 2026-10-28 | three scripted prior-report searches: 59 queries (225 distinct hits), 51 queries (216 hits), and 40 queries plus 20 manual GitHub searches | no report; #15936 is context only |
| GitHub manual (apache/lucene, apache/solr, OpenSearch, Elasticsearch) and JIRA text search | 2026-10-28 | "you'll get 7 groups", "7 groups back", topNGroups, groupOffset, "grouping package-info", GroupingSearch offset | 0 hits for the phrases; topNGroups in JIRA only LUCENE-3778 |
| Review threads and issue comments | 2026-10-28 | #15936, #16385, #16508, #16292, #15574 and all 23 comments of LUCENE-1421 read | no comment mentions the bullet |
| Open PRs | 2026-10-28 | the changed-file lists of all 414 open apache/lucene PRs | none touches the grouping package-info.java |
| Refresh | 2026-11-01 | GitHub search and API: issues and PRs updated since 2026-10-28; commits on the file | nothing about grouping offsets; last commit on the file still #16292 (2026-10-06) |
The sentence dates from LUCENE-1421 (3.2, 2011) and has disagreed with the page's own examples since. The managers shipped in 10.5.0 with page-size semantics; #16385 (10.6.0) made the page name only them, a direct contradiction. Not fixed upstream.
Versions tested5 rows
Direct contradiction introduced by a6e670d6ed1 "Remove unused public methods in AllGroupHeadsCollector (#16385)" (gaobinlong, reviewed by javanna, who asked for the package-info update; 2026-08-12, milestone 10.6.0; backport d729cbfd11f). It renamed the classes in the two-pass paragraph and turned the Block example into the manager form, and left the inputs list unchanged.
Older ingredients: the bullet and a two-pass example with groupOffset+topNGroups from LUCENE-1421 (9083cedb922, ee2154e4a39, 2011-05-14, 3.2); the Block example with groupOffset+topNGroups from LUCENE-3129 (375c1abdbf4 on trunk, 29ac743e238 on branch_3x, 2011-06-02; first released in 3.3, through the branch_3x commit); the managers from #15574 (636e8e9f587) and #15936 (d27d080d101), June 2026, first released in 10.5.0.
Not fixed upstream (a snapshot): upstream main f815e679bf6c (2026-11-01) still has the sentence; the last commit to the file is d148795b182 (#16292, 2026-10-06), on other lines. The parent of a6e670d6ed1 was not built: the commit changes no code the repro runs.
| Version or commit | Verdict | Result |
|---|---|---|
main 645d4fe14a20 | affected | minimal test fails with assertions on and off (gradle and standalone); the page names the managers and no plain collector |
10.6.0 (release jars; release commit 4ae946e5a9b) | affected | minimal test fails with assertions on and off; the hunt's test fails 2 of 3 |
| 10.5.0, 10.5.1, 10.5.2 | page named the plain collector | the managers already return 10 (the hunt's test on all three, the minimal test on 10.5.0), but the page named FirstPassGroupingCollector, for which the sentence holds (control returns 7); its Block example already returned 10 |
| 9.12.3, 10.4.0 | examples only | no managers; the page's Block example returns 10 against the bullet's 7 |
| 3.2, 3.3, 9.0.0 | text read | the same bullet next to examples that pass groupOffset+topNGroups (not run) |
The finder, an independent reproducer, a works-as-designed cross-model review (OpenAI GPT-5.6), a devil's-advocate verifier, the adjudication and a credibility audit all call it a real documentation defect; a reading-only refute review agreed on substance. This doc's gradle runs and release-jar runs confirm it on main, 10.6.0 and 10.5.0.
Checks and verdicts279 words
The cross-model reviews could only read the code (they could not run Java); no executing cross-model review was done. The adjudication required three corrections, applied here: the mismatch is long-standing, the managers behave this way since 10.5.0, and the fix names both plain collectors.
| Check | Verdict | What it found |
|---|---|---|
| finder | fails 2 of 3 | Both managers return 10 for (3, 10) on main and the 10.6.0 release commit, assertions on and off; the plain-collector control returns 7. |
| cross-model refute review | not real (procedural), agrees on substance | OpenAI GPT-5.6, reading only: could not run code, so "not reproduced"; agreed with the root cause and called it a genuine documentation defect. A separate re-run failed 2 of 3. |
| Independent reproducer | real (doc) | Own 6-test JUnit: 3 fail on main and 10.5.2, 3 controls pass; overlapping pages; the old Block example returns 10 on 9.12.3, 10.5.2 and main; corrected the version story. |
| Cross-model works-as-designed lens | real, high | OpenAI GPT-5.6, reading only: the intended-behaviour case covers the code, not the text; the fix should also name BlockGroupingCollector. |
| Devil's advocate | real (doc, minor) | Built the strongest intended-behaviour case and showed why it fails; re-runs fail on main and 10.5.2. |
| adjudication | confirmed, high | Re-ran the independent reproduction's test on main with assertions on and off (3 of 6 fail, controls pass); required the three corrections. |
| credibility audit | reproduces | Re-ran the finder's test with a fresh seed: 2 of 3 fail as expected. |
| this doc | fails; fix A validated | Gradle before and after fix A on 645d4fe14a20; minimal test on the 10.6.0 and 10.5.0 release jars, assertions on and off. |
The diagram as a still image, generated with AI and checked against the code. It is under 10 MB, the limit for a GitHub attachment.
- PNG 557 KB
Finding record, Lucene and upstream sources, repro files and diagram notes1,422 words
- Registry title
- The grouping package documentation says groupOffset 3 with topNGroups 10 returns 7 groups, but FirstPassGroupingCollectorManager and BlockGroupingCollectorManager, which the page names and shows since #16385 (10.6.0), return 10: for them topNGroups is the page size after the offset (documentation defect; the inputs list has also disagreed with the page's own example code since 3.2)
- Area
- grouping / package documentation (package-info.java inputs list vs the collector managers it names)
- Priority
- Low, rank 1235: This grouping doc defect, mostly new in 10.6.0, can silently lead users who follow the package prose into oversized, overlapping pages.
- Impact class
- minor
- Impact note
- Documentation only, in the optional grouping module: a developer who writes paging code from the package overview's inputs list and passes topNGroups = groupOffset + pageSize to FirstPassGroupingCollectorManager or BlockGroupingCollectorManager (the classes the 10.6 page names) gets pages that are larger than the page size and overlap the next page (page size 5 on 15 groups: the page at groupOffset 5 returns g05..g14 and the next page g10..g14); every returned group still matches the query, the code is correct and intended, GroupingSearch's own javadoc is right, and no scale or non-default setting is needed; the direct contradiction is in main and 10.6.0 (released 2026-10-29) only, while the inputs list has disagreed with the page's own example code in every release since 3.2 and the managers have behaved this way since 10.5.0.
- Summary
- You read the grouping package overview, which says topNGroups is 'how many top groups to keep' and that groupOffset 3 'means you'll get 7 groups back (assuming topNGroups is 10)', and which names FirstPassGroupingCollectorManager as the first pass and builds BlockGroupingCollectorManager(groupSort, groupOffset, topNGroups, ...), so you expect groupOffset 3 with topNGroups 10 to return 7 groups. Instead both managers, like GroupingSearch, treat topNGroups as the number of groups to return after the offset and return 10 groups (ranks 4 to 13). The '7 groups' sentence is right only for the plain FirstPassGroupingCollector and BlockGroupingCollector, which the page named until #16385 rewrote it for the managers, so paging code written from the sentence gets oversized, overlapping pages.
- Introduced
- a6e670d6ed1b4342accf75aed0d63ea11fde6645 (#16385 'Remove unused public methods in AllGroupHeadsCollector', by gaobinlong, reviewed and approved by javanna, who asked for the package-info update; milestone 10.6.0; backport d729cbfd11f in branch_10x and branch_10_6) [read: the commit rewrote the two-pass paragraph to name FirstPassGroupingCollectorManager and TopGroupsCollectorManager instead of the collectors, and the single-pass example from new BlockGroupingCollector(groupSort, groupOffset+topNGroups, ...) to new BlockGroupingCollectorManager(groupSort, groupOffset, topNGroups, ...), and left the inputs list unchanged; exec at the version boundary: on 10.5.2 the page names FirstPassGroupingCollector, for which the sentence holds (control returns 7), and on the branch_10_6 head 4ae946e5a9b (10.6.0, released 2026-10-29) the named managers return 10; the commit's parent was not built, because the commit changes no code that the repro runs]. Older ingredients [read: git log -S, blame]: the inputs-list sentence and a two-pass example that already passed groupOffset+topNGroups come from LUCENE-1421 (9083cedb922 on trunk, ee2154e4a39 on branch_3x, by mikemccand, 2011-05-14; first released in 3.2); the BlockGroupingCollector example with groupOffset+topNGroups from LUCENE-3129 (375c1abdbf4, 2011-06-02; 3.3); the managers' page-size semantics from #15574 (636e8e9f587, 2026-06-03) and #15936 (d27d080d101, 2026-06-15, where the author chose groupOffset+topNGroups in review to match GroupingSearch), both first released in 10.5.0. (2026-08-12 (author date of a6e670d6ed1, 21:16:52 +0800 = 13:16:52Z, merged the same minute; backport d729cbfd11f committed 2026-08-12 15:27:39 +0200); older ingredients 2011-05-14 (LUCENE-1421) and 2011-06-02 (LUCENE-3129))
- First affected version
- 10.6.0, released 2026-10-29 (branch_10_6 head 4ae946e5a9b, tag releases/lucene/10.6.0) for the direct contradiction with the classes the page names [exec + read]. Not in this form in 10.5.0-10.5.2, whose page names the collectors, although the managers already return 10 there [exec]. The weaker form, the inputs list disagreeing with the page's own example code (which passed groupOffset+topNGroups and so returns 10 groups for topNGroups 10), is in every release since 3.2 [exec with the pre-10.6 Block example on 9.12.3, 10.4.0 and 10.5.2; text read on 3.2, 3.3, 9.0.0, 9.12.3, 10.4.0, 10.5.0 and 10.5.2].
- Affected versions
- Direct contradiction (the page names the managers): main a9779ef3c193 [exec: the finder's JUnit fails 2 of 3 with -ea and -da, seed DEADBEEF, the collector control passes; the independent reproduction's JUnit fails 3 of 6 with -ea and -da, seeds C0FFEE and the adjudication's 5EED1234, its 3 controls pass; the intended-behaviour check re-ran the finder's JUnit with -ea and -da; re-run -da, seed 5EED3870, 2 of 3], the branch_10_6 head 4ae946e5a9b (10.6.0, released 2026-10-29) [exec, -ea and -da], branch_10x (10.7.0-SNAPSHOT, contains d729cbfd11f) [read]; upstream main 645d4fe14a20 (2026-10-28) still has the sentence [read: the last commit to the file is d148795b182, #16292, 2026-10-06, other lines]. Long-standing mismatch between the inputs list and the page's own example code: 3.2 to 10.5.2 and still on 10.6/main [exec: the independent reproduction's pre-10.6 BlockGroupingCollector example returns 10, not 7, on 9.12.3, 10.4.0, 10.5.2, 10.6.0 (branch_10_6 head) and main; read on 3.2, 3.3 and 9.0.0]. The managers' behaviour, unchanged since they shipped: 10.5.0, 10.5.1 and 10.5.2 return 10 for (3, 10) [exec]; 9.12.3 and 10.4.0 have no managers (the JUnit does not compile) [exec]. All runs JDK 25.0.3 on Linux aarch64.
- Status
- documented (brief, checks and a per-finding doc exist)
- Verification
- Confirmed by AI-run checks: an independent repro, a cross-model review that disputed it (the adjudication ruled it real), an intended-behaviour check, a prior-report search and a skeptical audit; not yet triaged by a person.
- Found by
- non-formal: code reading
- Upstream and prior reports
- none found (searched 2026-10-28): three scripted prior-report searches (the follow-up's 59 queries over GitHub issues and PRs, commits, downstream repositories, dev@/java-user@/issues@/commits@ lists, the JIRA archive and the web, 225 distinct hits; the finder's 51 queries, 216 hits; the intended-behaviour check's 40 queries plus 20 manual GitHub searches), manual GitHub searches over apache/lucene, apache/solr, OpenSearch and Elasticsearch (exact phrase "you'll get 7 groups": 0), JIRA text searches ('"7 groups back"': 0; topNGroups: only LUCENE-3778), all 23 comments of LUCENE-1421, the review threads of #15936, #16385, #16508, #16292 and #15574, and the changed-file lists of all 414 open apache/lucene PRs (none touches grouping package-info.java). Related, not reports: #15936 (the review thread where the managers' page-size semantics were chosen on purpose), #16385 (the rewrite that made the contradiction direct; no review comment mentions the bullet), #16508 and #16292 (other sentences of the same page), #4851 (LUCENE-3778, GroupingSearch's offset/limit), #15995 (a different withinGroupOffset code bug). Not fixed upstream: upstream main 645d4fe14a20 (2026-10-28) still has the sentence. Refresh 2026-10-28 (GitHub search and REST API): none of the 72 apache/lucene PRs updated since 2026-10-26 touches lucene/grouping; 'topNGroups groupOffset' finds only #4851; '"7 groups back"' finds nothing; no new commit on the file.
- Lucene commit tested
- main
645d4fe14a20: the minimal test through gradle (2026-11-01) and standalone (2026-10-29), assertions on and off; fix A in a scratch worktree (format check, javadoc build, the whole grouping test module); the minimal test on the 10.6.0 and 10.5.0 release jars, assertions on and off (2026-11-01). The hunt's and verifiers' tests ran on maina9779ef3c193(same grouping sources), the 10.6.0 release commit4ae946e5a9band 10.5.0 to 10.5.2; the page's pre-10.6 Block example on 9.12.3, 10.4.0, 10.5.2, 10.6.0 and main.
- Finding record
a Lucene main checkout lucene/grouping/src/java/org/apache/lucene/search/grouping/package-info.java:37-41 (inputs list), :49-51 (first pass = FirstPassGroupingCollectorManager), :129-137 (BlockGroupingCollectorManager example); FirstPassGroupingCollectorManager.java:66-67 and :82-83 (@param), :106-111 (groupOffset + topNGroups), :125 (SearchGroup.merge(allGroups, groupOffset, topNGroups, groupSort)); BlockGroupingCollectorManager.java:87-88 (@param), :141-143, :158-159 (mergeBlockGroups with groupOffset, topNGroups); GroupingSearch.java:123 (@param groupLimit); FirstPassGroupingCollector.java:72, :86 and BlockGroupingCollector.java:204 ('How many top groups to keep') [read by the registration check]a Lucene clone releases/lucene/10.5.2 package-info.java:40, :50, :128 (names FirstPassGroupingCollector; Block example with groupOffset+topNGroups); in the same clone, d729cbfd11f is an ancestor of 4ae946e5a9b [read by the registration check]
- The finding in 30 seconds
lucene/grouping/src/java/org/apache/lucene/search/grouping/package-info.java:37-41, :49-52, :129-137 (at 645d4fe14a20)lucene/grouping/src/java/org/apache/lucene/search/grouping/FirstPassGroupingCollectorManager.java:66-67, :82-83, :106-111, :125 (at 645d4fe14a20)lucene/grouping/src/java/org/apache/lucene/search/grouping/SearchGroup.java:278-311 (at 645d4fe14a20)lucene/grouping/src/java/org/apache/lucene/search/grouping/BlockGroupingCollectorManager.java:87-88, :141-144, :158-159 (at 645d4fe14a20)lucene/grouping/src/java/org/apache/lucene/search/grouping/GroupingSearch.java:122-123, :148 (at 645d4fe14a20)lucene/grouping/src/java/org/apache/lucene/search/grouping/FirstPassGroupingCollector.java:72, :86, :133-136; BlockGroupingCollector.java:204, :256-261 (at 645d4fe14a20)
- Who is affected
lucene/grouping/src/java/org/apache/lucene/search/grouping/FirstPassGroupingCollectorManager.java:50, :106-111; BlockGroupingCollectorManager.java:68 (at 645d4fe14a20)
- Why it looks unintended
lucene/grouping/src/test/org/apache/lucene/search/grouping/TestGrouping.java:676, :680, :1110-1112, :1300-1309 (at 645d4fe14a20)lucene/grouping/src/java/org/apache/lucene/search/grouping/GroupingSearch.java:123 (at 645d4fe14a20)lucene/grouping/src/test/org/apache/lucene/search/grouping/TestGrouping.java:173-174; TestGroupingSearch.java:125-126 (hit counts; at 645d4fe14a20)
- Suggested fix
lucene/grouping/src/java/org/apache/lucene/search/grouping/TopGroupsCollectorManager.java:50-56 (withinGroupOffset only), AllGroupHeadsCollectorManager.java, TermGroupFacetCollectorManager.java (no groupOffset or topNGroups; at 645d4fe14a20)
- History
a Lucene clone tag releases/lucene/10.6.0 = 4ae946e5a9b, contains d729cbfd11f
- Diagrams
lucene/grouping/src/java/org/apache/lucene/search/grouping/FirstPassGroupingCollectorManager.java:110, :125; BlockGroupingCollectorManager.java:143, :159 (at 645d4fe14a20)
- TestGroupingPackageDocOffset.java:
repro/LFV-961/src/org/apache/lucene/search/grouping/TestGroupingPackageDocOffset.java(56 lines)
assets/LFV-961/bug.png(557 KB)
