Sequential DrillSideways lets a top-N hit collector prune the shared base query, so sideways facet counts silently miss documents

The finding in 30 seconds

Drill sideways keeps a facet useful after the user picks a value: after a click on red, the color facet still lists blue 75, so the user can switch.

Two things to know first

Drill sideways: for each drilled-down dimension, DrillSideways counts facets as if that dimension's own filter were not applied. Without an executor it does this in one pass: one iteration over the base query feeds the hit collector and every sideways counter.
A top-N collector with a total-hits threshold t counts hits exactly up to t. After that it may call setMinCompetitiveScore on the scorer it was given, which tells that scorer it may skip documents that cannot enter the top N. A constant-score filter can then skip everything left.

A diagram on a light card: on the left a search page for body:a filtered to color = red shows the top 10 results and a color sidebar with red 75 and blue 75, labelled as sideways counts that ignore the color filter; on the right the base query's matches (docs 0-149) feed one shared pass, outlined in red as the place of the bug, which splits into a hit collector that keeps the 10 best and a sideways counter that counts every match by color, and a red arrow from the hit collector back to the shared pass says that after 11 hits it asks for only scores above 1.0 (setMinCompetitiveScore); the bottom panels show the expected result in orange, every one of the 150 matches counted for red 75 and blue 75, beside the actual result in blue with a red cross, the pass stopping after doc 20 for red 11 and blue 10. — Sequential DrillSideways reads the base query's matches once for both the hits and the sideways counts, so when the hit collector asks to skip low-scoring documents the shared pass stops and the counts come out too low. Generated with AI, checked against the code.

Diagram: the sequential DrillSideways runs one iteration over the base query (a filter on shoes) and feeds each document to the top-10 hit collector and to the sideways color counters. After the hit collector has counted past its threshold it calls setMinCompetitiveScore on the shared base scorer, the iteration stops, and the color counts end too low (red 11, blue 10 instead of 75 and 75 in the repro). — One base-query iteration feeds both the top-N hit collector and the sideways facet counters; when the hit collector raises its minimum competitive score, the shared iterator skips the rest, so the counters never see those documents. Select the still to open it at full size. Generated with AI, checked against the code.

The finding, step by step

Index 4,000 products, half red and half blue; 150 of them are shoes, 75 red and 75 blue. Filter on shoes, drill down on color=red, and run the sequential DrillSideways (no executor) with TopScoreDocCollectorManager(10, 10): top 10 hits, hit count exact up to 10.
You expect the color facet to ignore its own drill-down and count every shoe: red 75, blue 75. An unlimited threshold, or the concurrent DrillSideways, gives exactly that.
Instead you get red 11, blue 10. At the 11th hit, one past its threshold, the hit collector raises its minimum competitive score. DrillSideways handed it the base query's own scorer, so the filter's iterator stops after doc 20 for everyone, and the facet counters never see shoes 21-149.
Nothing fails, and the top 10 hits are the same as with an exhaustive collector. With IndexSearcher's default threshold, 1000, the drilled value stops at 1001: 1001 instead of 2250 on 48,000 documents.

Who is affected

The sequential DrillSideways with a caller's finite-threshold top-N manager (1000 included), via the 3-arg search or a null drill-down facets hook: TopScoreDocCollectorManager(n, t) on a filter-style or minShouldMatch > 1 base, a relevance-sorted TopFieldCollectorManager only on the latter; a narrow base and more than t drill-down hits. The default 2-arg search is safe since 9.12.0.

Mechanism, terms and numbers568 words

DrillSidewaysQuery.createWeight builds the base query's weight with the hit collector's score mode (DrillSidewaysQuery.java:129 on main 5adff78bb168), which is TOP_SCORES for a TopScoreDocCollectorManager or relevance-sorted TopFieldCollectorManager with a finite total-hits threshold. In the query-first strategy (DrillSidewaysScorer.java:130), doQueryFirstScoring hands the base Scorer itself to the hit collector and the sideways collectors (:199-200). Once more documents than its threshold have matched every dimension, the hit collector's setMinCompetitiveScore reaches the shared base scorer: with TopScoreDocCollectorManager a constant-score base empties its iterator and a minShouldMatch > 1 disjunction skips documents; a relevance-sorted TopFieldCollectorManager asks only for the bottom score itself, which prunes the minShouldMatch > 1 case only. The sideways FacetsCollectors never see the rest, so drill-sideways counts are silently too low (red 11, blue 10 instead of 75 and 75 in the repro; 1001 instead of 2250 at threshold 1000), while the top hits matched an exhaustive run in every executed case. The other two scoring strategies and the concurrent DrillSideways are exact. Introduced by LUCENE-10030 (8.10.0 / 9.0.0); affected through 10.6.0 and main. The fix hands the collectors a FilterScorable, as core's MultiCollector does; building the base weight with an exhaustive score mode is an optional extra.

The code (main `5adff78bb168`)

The base weight gets the hit collector's score mode. searchSequentially runs searcher.search(dsq, drillDownCollectorManager) (DrillSideways.java:467), and DrillSidewaysQuery.createWeight passes that score mode on: baseQuery.createWeight(searcher, scoreMode, boost) (DrillSidewaysQuery.java:129). The drill-down weights always use COMPLETE_NO_SCORES (:133-134). TopScoreDocCollector.scoreMode() is TOP_SCORES unless the threshold is Integer.MAX_VALUE (TopScoreDocCollector.java:58-60); IndexSearcher.search(Query, int) uses 1000 (IndexSearcher.java:100, :599).
Query-first scoring shares the base scorer. When scoreSubDocsAtOnce() is set or the base cost is below a tenth of the drill-down cost (DrillSidewaysScorer.java:130; in the repro 150 vs 2,000), doQueryFirstScoring runs collector = ScoreCachingWrappingScorer.wrap(collector); setScorer(collector, baseScorer); (:199-200), and setScorer gives the same scorable to every sideways leaf collector (:746-751). ScoreCachingWrappingScorer forwards setMinCompetitiveScore (ScoreCachingWrappingScorer.java:93-96).
The hit collector prunes. After more than t hits (only documents that match every dimension reach it, DrillSidewaysScorer.java:714-719), TopScoreDocCollector calls scorer.setMinCompetitiveScore(nextUp(topScore)) (TopScoreDocCollector.java:155-162). A constant-score base has the same score for every document, so ConstantScoreScorer replaces its iterator with an empty one (ConstantScoreScorer.java:149-153); a WANDScorer (minShouldMatch > 1) raises its bound and skips (WANDScorer.java:258-267). A relevance-sorted TopFieldCollector passes the bottom entry's own score instead (TopFieldCollector.java:337-343), which ConstantScoreScorer ignores (it needs minScore > score, :150), so with it only a minShouldMatch > 1 base is pruned.
The sideways counters starve. The same iteration drives collectHit and collectNearMiss for the sideways collectors (DrillSidewaysScorer.java:697-735), so they stop receiving documents too. In the repro the iteration ends right after the 11th red shoe, doc 20: 11 red and 10 blue documents were counted.
The other paths are exact. The drill-down-advance and union strategies give every collector a private Score scorable (DrillSidewaysScorer.java:336, :548, class at :753-764) that keeps Scorable's no-op setMinCompetitiveScore (Scorable.java:45-52); the concurrent DrillSideways runs separate searches. So the same query can be exact or wrong depending on an internal cost heuristic: an independent test got 1500 with the union strategy and 11 with scoreSubDocsAtOnce() for the same base and collector.

Counts can only be too low: every document still collected passes the base query, its filters, the live docs and the other drill-down dimensions.

Exposure and likelihood521 words

Who is hit. The sequential DrillSideways (no executor) plus a caller-supplied top-N hit collector manager with a finite t (1000 included): TopScoreDocCollectorManager(n, t), or, with a minShouldMatch > 1 base only, TopFieldCollectorManager(Sort.RELEVANCE, n, t). Routes on main: the 3-arg search(DrillDownQuery, CollectorManager, List), documented as usable "with any CollectorManagers" (DrillSideways.java:399-400), and the 2-arg search when createDrillDownFacetsCollectorManager() returns null, documented as valid (:133-135; the hit manager is then used alone, :338).
Which queries. Base scorers that react to the hit collector's minimum competitive score: with TopScoreDocCollectorManager, filter-only booleans, ConstantScoreQuery, point or term ranges and minShouldMatch > 1 disjunctions; with a relevance-sorted TopFieldCollectorManager, only minShouldMatch > 1 disjunctions (it passes the bottom score itself, which a constant-score scorer ignores; with it the filter and ConstantScoreQuery bases gave 75/75). On main a plain scoring TermQuery or plain SHOULD disjunction is not affected (verifier controls). The base must cost less than a tenth of the drilled value (the common "filter, then pick a broad facet value" case), or scoreSubDocsAtOnce() is overridden.
Scale. Pruning starts once more than t documents match every drill-down dimension, since only those reach the hit collector (DrillSidewaysScorer.java:714-719; near misses go to the sideways collector only, :730-735): with the repro's 150 shoes, 75 of them red, t = 74 gives 75/74 and t = 75, 100 or 149 give 75/75. With a constant-score base the loss is deterministic, and in one segment the drilled value ends at t + 1: 11 for t = 10; 1001 instead of 2250 (48,000 docs) and 1001 instead of 2000 (60,000 docs) at t = 1000. With about 12 segments and an IntPoint range base: 1171 instead of 3000. With a minShouldMatch > 1 base the loss depends on the data and the release: the history test's minShouldMatch=2 case is exact on 9.12.x and 10.0.0, gives 50/50 instead of 75/75 on 10.1.0-10.3.0 and 21/19 from 10.4.0.
Silent. No exception or log line; the counts look plausible and the top hits matched the exhaustive run in every executed case, with assertions on or off.
Default vs opt-in. Default codec and IndexWriterConfig and the default (sequential) DrillSideways mode; the test turns the query cache off, which is main's default (IndexSearcher.java:84; 10.6.0 and 9.12.3 default to an LRUQueryCache, which the test also turns off). The trigger is the caller's finite-threshold collector on the 3-arg path or the null hook. In 8.10.0-9.11.x the plain default search(DrillDownQuery, Collector) and the 2-arg manager search with default hooks were affected too, and drill-down counts were low as well. Since 9.12.0 the default 2-arg path wraps hits and facets in MultiCollectorManager, which forces a COMPLETE mode.
Workarounds. An Integer.MAX_VALUE threshold; the concurrent DrillSideways; or, as the 3-arg search's javadoc Note suggests (DrillSideways.java:402-404), a MultiCollectorManager holding the hit manager together with a FacetsCollectorManager, which then uses a COMPLETE mode (MultiCollector.java:104-121) and was exact in a verifier's control. A MultiCollectorManager around the hit manager alone is still affected.
Likelihood: plausible. No public report of this mechanism since 8.10.0 (2021).

Minimal repro

One JUnit test, public API only: 4,000 documents, a filter on 150 shoes, a drill-down on color=red, and the sequential DrillSideways with TopScoreDocCollectorManager(10, 10). It fails on main with assertions on and off, and on 10.6.0 and 9.12.3 with assertions on, and passes with the fix.

TestDrillSidewaysTopScoresHitCollector.java (one test, lucene/facet)91 lines

/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.apache.lucene.facet;

import java.util.List;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.document.StringField;
import org.apache.lucene.facet.sortedset.DefaultSortedSetDocValuesReaderState;
import org.apache.lucene.facet.sortedset.SortedSetDocValuesFacetCounts;
import org.apache.lucene.facet.sortedset.SortedSetDocValuesFacetField;
import org.apache.lucene.facet.sortedset.SortedSetDocValuesReaderState;
import org.apache.lucene.index.DirectoryReader;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.index.Term;
import org.apache.lucene.search.BooleanClause.Occur;
import org.apache.lucene.search.BooleanQuery;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.TermQuery;
import org.apache.lucene.search.TopDocs;
import org.apache.lucene.search.TopScoreDocCollectorManager;
import org.apache.lucene.store.Directory;
import org.apache.lucene.tests.util.LuceneTestCase;
import org.apache.lucene.tests.util.TestUtil;

public class TestDrillSidewaysTopScoresHitCollector extends LuceneTestCase {

  public void testSidewaysCountsDoNotDependOnHitCollectorThreshold() throws Exception {
    FacetsConfig config = new FacetsConfig();
    try (Directory dir = newDirectory()) {
      IndexWriterConfig iwc = new IndexWriterConfig().setCodec(TestUtil.getDefaultCodec());
      try (IndexWriter w = new IndexWriter(dir, iwc)) {
        // 4000 docs, half red and half blue; the first 150 are shoes (75 red, 75 blue)
        for (int i = 0; i < 4000; i++) {
          Document doc = new Document();
          doc.add(new SortedSetDocValuesFacetField("color", i % 2 == 0 ? "red" : "blue"));
          if (i < 150) {
            doc.add(new StringField("category", "shoes", Field.Store.NO));
          }
          w.addDocument(config.build(doc));
        }
        w.forceMerge(1);
      }
      try (DirectoryReader reader = DirectoryReader.open(dir)) {
        IndexSearcher searcher = new IndexSearcher(reader);
        searcher.setQueryCache(null); // as on main by default
        SortedSetDocValuesReaderState state =
            new DefaultSortedSetDocValuesReaderState(reader, config);

        // filter on shoes, drill down on color=red
        Query shoes =
            new BooleanQuery.Builder()
                .add(new TermQuery(new Term("category", "shoes")), Occur.FILTER)
                .build();
        DrillDownQuery ddq = new DrillDownQuery(config, shoes);
        ddq.add("color", "red");

        // no executor: the sequential DrillSideways; top 10 hits, count hits up to 10
        DrillSideways ds = new DrillSideways(searcher, config, state);
        DrillSideways.Result<TopDocs, FacetsCollector> result =
            ds.search(
                ddq,
                new TopScoreDocCollectorManager(10, 10),
                List.of(new FacetsCollectorManager()));

        // the color dimension ignores its own drill-down: every shoe counts, 75 red and 75 blue
        FacetsCollector sideways = result.drillSidewaysResults().get(0);
        SortedSetDocValuesFacetCounts counts = new SortedSetDocValuesFacetCounts(state, sideways);
        int red = counts.getSpecificValue("color", "red").intValue();
        int blue = counts.getSpecificValue("color", "blue").intValue();
        assertEquals("sideways color counts red/blue", "75/75", red + "/" + blue);
      }
    }
  }
}

Command

cd $LUCENE
cp $REPRO/TestDrillSidewaysTopScoresHitCollector.java lucene/facet/src/test/org/apache/lucene/facet/
./gradlew :lucene:facet:test --tests org.apache.lucene.facet.TestDrillSidewaysTopScoresHitCollector -Ptests.seed=DEADBEEF -Ptests.asserts=true
# the same failure with -Ptests.asserts=false

Expected: The sideways color counts include every shoe: red 75, blue 75.
Observed: ComparisonFailure: sideways color counts red/blue expected:<[75/75]> but was:<[11/10]>.

All files of this repro: repro/LFV-789/

More files, environment, full output136 words

Single thread, deterministic: the codec is pinned to the default and the query cache is off, as on main by default (10.6.0 and 9.12.3 default to an LRUQueryCache; the test turns it off there too). The test goes in lucene/facet/src/test/org/apache/lucene/facet/ and compiles unchanged against 10.6.0 and 9.12.3.

Controls of the same file: TopScoreDocCollectorManager(10, Integer.MAX_VALUE) (a one-line change) gives 75/75, and the concurrent DrillSideways (an executor passed to the constructor) with threshold 10 gives 75/75.

Regression test: this same test. It passes with fix A alone and with A plus the optional base score mode.

Environment

lucene	`main 5adff78bb168 (12.0.0-SNAPSHOT); also fails on 10.6.0 and 9.12.3 (assertions on)`
jdk	`OpenJDK 25.0.3 (build 25.0.3+9-LTS)`
os	`Linux, aarch64`

Observed output

TestDrillSidewaysTopScoresHitCollector > testSidewaysCountsDoNotDependOnHitCollectorThreshold FAILED
    org.junit.ComparisonFailure: sideways color counts red/blue expected:<[75/75]> but was:<[11/10]>
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:1ED870717072830B]:0)
        at app/junit@4.13.2/org.junit.Assert.assertEquals(Assert.java:117)
        at app//org.apache.lucene.facet.TestDrillSidewaysTopScoresHitCollector.testSidewaysCountsDoNotDependOnHitCollectorThreshold(TestDrillSidewaysTopScoresHitCollector.java:87)

Why it looks unintended

Sideways collectors are documented to see every hit matching the other dimensions, and the 3-arg search "can be used with any CollectorManagers".
A threshold only affects the hit count; core's MultiCollector ignores setMinCompetitiveScore so that "the other collector still sees all hits".
The same search is exact in the concurrent mode and in the other two scoring strategies.

The full argument257 words

The sideways contract. "Drill sideways counts include alternative values/aggregates for the drill-down dimensions so that a dimension does not disappear after the user drills down into it" (DrillSideways.java:50-52). Each sideways collector "exposes the hits that match on all DrillDownQuery dimensions, but treating their corresponding sideways dimension as optional" (:272-274).
Any collector manager is allowed. "This method can be used with any CollectorManagers" (:399-400); for the drill-down facets hook, "Returning null is valid if no drill down facet collection is needed" (:133-135).
The modes differ in speed, not results. The concurrent implementation "may be a little faster but does duplicate work"; the standard one "may be a little slower but avoids duplicate computations and query processing" (:57-62).
A threshold only affects the hit count. "If the query matches more than totalHitsThreshold hits then its hit count will be a lower bound" (TopScoreDocCollectorManager.java:50-54).
Core decided the same question. MultiCollector ignores setMinCompetitiveScore "so that if we wrap two collectors and one of them wants to skip low-scoring hits, then the other collector still sees all hits" (MultiCollector.java:195-202, LUCENE-8382, #9429), and uses a COMPLETE mode when its collectors' modes differ (:104-121).
History. LUCENE-10030 (#11069, PR #217) exposed the base scorer so that scores are computed lazily; its issue thread, PR description and CHANGES entry discuss only the cost of score(). Before it, every strategy gave collectors a private scorable that ignored the call. No facet test passes a TOP_SCORES hit collector to DrillSideways.

The devil's-advocate check241 words

The strongest case for "works as designed": TOP_SCORES explicitly permits skipping ("its iterator may safely ignore all documents whose score is less than the given minScore", Scorable.java:45-51); DrillSideways is @lucene.experimental (DrillSideways.java:69); its own top-N helpers use Integer.MAX_VALUE thresholds (:200-201, :232-233); and upstream knowingly lets an explicit CollectionTerminatedException from the hit collector cut sideways collection short (the TODO at DrillSidewaysScorer.java:149-151, TestDrillSideways.testCollectionTerminated), while #15220 asks for incomplete counts as an opt-in; and the 3-arg search's own javadoc Note recommends MultiCollectorManager for collecting hits and facets (DrillSideways.java:402-404), which makes the search exact.

Why it fails: the permission belongs to the collector that asked for TOP_SCORES; the sideways collectors (by default FacetsCollector, COMPLETE_NO_SCORES) never asked to skip, and core's MultiCollector blocks exactly this cross-collector pruning. Raising a minimum score is not a termination request; in #15220 a maintainer (gsmiller) described the design as "each wrapped collector can independently terminate, but those terminations don't impact the ability for other sibling collectors to continue". The result also depends on an internal cost heuristic and on the executor setting, and no javadoc, test or commit requires an exhaustive hit collector; the javadoc Note is advice on collecting facets for the whole query, and a MultiCollectorManager around the hit manager alone is still affected. Maintainers could still choose to document "the hit collector must be exhaustive" instead; that would contradict the 3-arg search's javadoc as written.

Suggested fix

Recommended FilterScorable for the collectors (the minimal fix)

In doQueryFirstScoring, hand the collectors new FilterScorable(baseScorer), which ignores setMinCompetitiveScore as MultiCollector does. Validated with gradle: the repro passes with assertions on and off, and 42 DrillSideways tests pass. Building the base weight with COMPLETE (or COMPLETE_NO_SCORES) on top is optional; with it all 307 facet tests and :lucene:facet:check -x test pass.

All options and trade-offs3 options · 802 words

Both code options were validated on 2026-11-03 with OpenJDK 25.0.3, seed DEADBEEF, in a clean scratch worktree of 5adff78bb168, and, for the finder's two-test repro, with the patched classes compiled over the main jars (a filter base with TopScoreDocCollectorManager(10, 10) and a minShouldMatch=2 base with TopFieldCollectorManager(Sort.RELEVANCE, 10, 10)).

The score-mode change alone is not a fix: ConstantScoreScorer then ignores the call, but WANDScorer still receives it, so the minShouldMatch=2 test fails with "minCompetitiveScore can only be set for ScoreMode.TOP_SCORES, but got: COMPLETE" under -ea and counts 12 instead of 100 under -da.

The open proposals #11128 (competitive iterators in DrillSidewaysScorer) and #11076 (one scoring path through doQueryFirstScoring) would widen the exposure if implemented without this guard.

FilterScorable for the collectors (the minimal fix) (validated by the checks) (recommended)

DrillSidewaysScorer.doQueryFirstScoring hands the collectors new FilterScorable(baseScorer); its setMinCompetitiveScore is a no-op (FilterScorable.java:27-28), as in MultiCollector. This alone fixes every tested case.

--- a/lucene/facet/src/java/org/apache/lucene/facet/DrillSidewaysScorer.java
+++ b/lucene/facet/src/java/org/apache/lucene/facet/DrillSidewaysScorer.java
@@ -25,6 +25,7 @@
 import org.apache.lucene.index.LeafReaderContext;
 import org.apache.lucene.search.BulkScorer;
 import org.apache.lucene.search.DocIdSetIterator;
+import org.apache.lucene.search.FilterScorable;
 import org.apache.lucene.search.LeafCollector;
 import org.apache.lucene.search.Scorable;
 import org.apache.lucene.search.ScoreCachingWrappingScorer;
@@ -197,7 +198,9 @@
   private void doQueryFirstScoring(Bits acceptDocs, LeafCollector collector, DocsAndCost[] dims)
       throws IOException {
     collector = ScoreCachingWrappingScorer.wrap(collector);
-    setScorer(collector, baseScorer);
+    // The sideways collectors need every hit of the base query, so no collector may make the
+    // shared base scorer skip documents: FilterScorable ignores setMinCompetitiveScore.
+    setScorer(collector, new FilterScorable(baseScorer));
 
     // Specialize the single-dim use-case as we have a more efficient implementation for that:
     if (dims.length == 1) {

Trade-offs

Smallest change. Scores stay lazy (FilterScorable.score() delegates), so the gain of LUCENE-10030 is kept. Because setScorer gives the same scorable to the sideways collectors, a pruning sideways collector manager is covered as well (by reading, not tested).
FilterScorable.getChildren() reports the base scorer as a "FILTER" child, while the other strategies' Score reports it as "MUST" (DrillSidewaysScorer.java:760-763); override getChildren() if that matters.

What the checks validated

The repro fails stock and passes with the fix, assertions on and off (gradle).
TestDrillSideways, TestParallelDrillSideways, TestDrillDownQuery and the repro: 42 tests, 0 failures, assertions on and off.
The finder's two tests pass 2 of 2 with -ea and -da (patched classes over the main jars); earlier, a verifier's one-line FilterScorable probe made all 7 of its tests pass, the threshold-1000 case included.

Optional extra: also build the base weight with an exhaustive score mode (validated by the checks)

Fix A plus DrillSidewaysQuery.createWeight building the base weight with COMPLETE when scores are needed, else COMPLETE_NO_SCORES, as MultiCollector.scoreMode() does for mixed modes. Defense in depth: it states the invariant in the weight, but fixes no tested case that A leaves open, so it is a choice for reviewers.

--- a/lucene/facet/src/java/org/apache/lucene/facet/DrillSidewaysQuery.java
+++ b/lucene/facet/src/java/org/apache/lucene/facet/DrillSidewaysQuery.java
@@ -126,7 +126,10 @@
   @Override
   public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float boost)
       throws IOException {
-    final Weight baseWeight = baseQuery.createWeight(searcher, scoreMode, boost);
+    // The sideways collectors need every hit of the base query: never let the base weight prune.
+    final ScoreMode baseScoreMode =
+        scoreMode.needsScores() ? ScoreMode.COMPLETE : ScoreMode.COMPLETE_NO_SCORES;
+    final Weight baseWeight = baseQuery.createWeight(searcher, baseScoreMode, boost);
     final Weight[] drillDowns = new Weight[drillDownQueries.length];
     for (int dim = 0; dim < drillDownQueries.length; dim++) {
       drillDowns[dim] =
--- a/lucene/facet/src/java/org/apache/lucene/facet/DrillSidewaysScorer.java
+++ b/lucene/facet/src/java/org/apache/lucene/facet/DrillSidewaysScorer.java
@@ -25,6 +25,7 @@
 import org.apache.lucene.index.LeafReaderContext;
 import org.apache.lucene.search.BulkScorer;
 import org.apache.lucene.search.DocIdSetIterator;
+import org.apache.lucene.search.FilterScorable;
 import org.apache.lucene.search.LeafCollector;
 import org.apache.lucene.search.Scorable;
 import org.apache.lucene.search.ScoreCachingWrappingScorer;
@@ -197,7 +198,9 @@
   private void doQueryFirstScoring(Bits acceptDocs, LeafCollector collector, DocsAndCost[] dims)
       throws IOException {
     collector = ScoreCachingWrappingScorer.wrap(collector);
-    setScorer(collector, baseScorer);
+    // The sideways collectors need every hit of the base query, so no collector may make the
+    // shared base scorer skip documents: FilterScorable ignores setMinCompetitiveScore.
+    setScorer(collector, new FilterScorable(baseScorer));
 
     // Specialize the single-dim use-case as we have a more efficient implementation for that:
     if (dims.length == 1) {

Trade-offs

A caller that relied on TOP_SCORES-specific behaviour of the base scorer no longer gets it. No benchmark was run.
Not a fix on its own: with only the score-mode change, WANDScorer still receives the call (an assertion failure under -ea, 12 instead of 100 under -da).

What the checks validated

The repro fails stock and passes with the fix, assertions on and off (gradle).
The whole :lucene:facet:test with assertions on: 307 tests, 0 failures, 7 skipped.
:lucene:facet:check -x test (formatting, ecj lint, forbidden APIs) passes with the test in place.
The finder's two tests pass 2 of 2 with -ea and -da (patched classes over the main jars).

Documentation only: require an exhaustive hit collector

State on the 3-arg search and on the null drill-down facets hook that the hit collector must not prune (threshold Integer.MAX_VALUE).

Trade-offs

Cheapest, but it contradicts "can be used with any CollectorManagers", keeps a silent failure mode for existing callers, and leaves the answer dependent on the cost heuristic. Not validated (no code change).

Downstream note

A product that shows drill-sideways counts while collecting hits with a finite-threshold top-N manager shows counts that are too low; every counted document still matches the user's query and filters, so nothing hidden is revealed.

How it was found

Code reading: an inventory of every place where a non-exhaustive score mode reaches a scorer whose matches feed more than one collector found that the sequential DrillSideways hands the base scorer to the hit collector. A differential JUnit test, the same search with an exhaustive collector and with the concurrent mode, confirmed it.

The method in full126 words

Method: non-formal (code reading, then an executable differential check), on main a9779ef3c193, 2026-10-27.

The inventory listed every site that passes a non-exhaustive ScoreMode to a weight whose matches feed more than one consumer. DrillSidewaysQuery.createWeight passes the hit collector's mode to the base weight, and doQueryFirstScoring hands that scorer to the hit collector and the sideways collectors.
A public-API JUnit test compared the sideways counts of the sequential DrillSideways with a TOP_SCORES hit collector against the same search with an exhaustive hit collector and against the concurrent DrillSideways; only the first was low.
Independent verifiers wrote their own tests from the claim, an adjudication re-ran them, a history sweep bisected the change to LUCENE-10030, and this doc's one-test repro was re-run on main 5adff78bb168.

Prior reports, history and checks

Prior reports

No report of this defect found in GitHub issues and PRs, the JIRA archive, the mailing lists, downstream projects or the web (199 queries, 2026-10-27; re-checked 2026-11-03). Related but different: #15220 and draft #15221 ask for an explicit CollectionTerminatedException to stop facet collection; #9429 fixed this class of problem in MultiCollector.

Searches4 searches

In the searched sources, no Lucene issue, PR, JIRA entry, list thread or commit message, and nothing in the searched downstream projects and web results, describes a TOP_SCORES hit collector's minimum competitive score pruning the base scorer that the sideways collectors share. Related items, each about a different defect:

#15220 (open, tflobbe, 2025-09), "DrillSideways and early termination": since the default path wraps the hit collector in MultiCollector, a hit collector that throws CollectionTerminatedException no longer stops facet collection; the reporter asks for stopping as an opt-in feature. It never mentions setMinCompetitiveScore, score modes, the base scorer or a hits threshold. Its 8.10 symptom, incomplete counts, is the same kind of result this bug gives (the 8.10.0-9.11.x default path was exposed to it), and the issue does not name the collector, but it attributes the counts to explicit termination and wants them: a related design question with a different, explicit mechanism, while this bug needs no exception at all. The maintainer reply (gsmiller) says sibling collectors continue by design and points to the 3-arg search with a custom MultiCollectorManager that "terminates all collection as soon as one collector terminates".
Draft PR #15221 (a failing test for that request) and #11040 (LUCENE-10001, make MultiCollector's termination handling configurable): the same opt-in termination request.
#12640: added the TODO in DrillSidewaysScorer.score about CollectionTerminatedException; with testCollectionTerminated it covers the explicit-exception case only.
#9429 (LUCENE-8382, fixed 2018): the precedent; MultiCollector stopped forwarding setMinCompetitiveScore so one collector cannot hide hits from another.
#11069 / PR #217 (LUCENE-10030): the introducing change; its discussion covers only the cost of score().
#12211 / #12212 ("DrillSideways may miss documents", fixed in 9.6.0): a two-phase matches() called twice. #11098 (LUCENE-10060): query caching bypassed DrillSidewaysScorer. Both different mechanisms.
#11128 (LUCENE-10090) and #11076 (LUCENE-10037): open proposals for the scorer; neither discusses sideways counts.

What was searched, when, and what turned up
Where	When	How	Found
GitHub issues, PRs, commits and code; Elasticsearch, OpenSearch, Solr; dev@, java-user@, issues@, commits@; JIRA archive; web	2026-10-27	199 queries (166 from the scripted prior-report search in three runs, 33 manual: 18 GitHub, 8 web, 3 list statistics, 3 JIRA, 1 open-PR scan); #15220, #15221, #11069 and #217 read in full with comments	only the related items above
Open PRs	2026-10-27	the changed files of 409 open apache/lucene PRs	#16707, draft #15221 and draft #16847 touch DrillSideways files; none changes createWeight or doQueryFirstScoring
Re-check (two independent passes)	2026-11-03	the related items re-read against this defect	related but about a different defect; nothing describes this one
Re-check for this doc (GitHub, commits, downstream, lists, JIRA)	2026-11-03	the scripted prior-report search again (45 queries), 10 manual GitHub searches including items updated since 2026-10-20, recent commits and open PRs on the two files; commits and open-PR files re-read	nothing new; no 2026 commit on DrillSidewaysScorer.java or DrillSidewaysQuery.java; of the open PRs touching DrillSideways files, #16847 (now ready for review) only changes maxDoc to maxLeafDoc in the other two strategies

History

Introduced by LUCENE-10030 (#217, 2021), which handed the hit collector the base scorer so scores are computed lazily: first released in 8.10.0 and 9.0.0. 8.9.0 passes, and reverting that one file fixes 8.10.0 and 9.0.0. Affected through 10.6.0 and main; not fixed upstream.

Versions tested9 rows

Introduced by LUCENE-10030, "Lazily evaluate score in DrillSidewaysScorer.doQueryFirstScoring" (issue #11069, PR #217 by gtroitskiy, merged by gsmiller): 61f8517000b3 on main (2021-07-26, first release 9.0.0) and the same change on branch_8x, c7210e494e16 (apache/lucene-solr#2542, 2021-08-01, first release 8.10.0). It removed the shared collector.setScorer(new ScoreAndDoc()) from score() and gave each strategy its own call: new ScoreAndDoc() in the other two and setScorer(collector, new ScoreCachingWrappingScorer(baseScorer)) in doQueryFirstScoring; ScoreAndDoc ignored setMinCompetitiveScore. Older preconditions: the base weight takes the caller's score mode since LUCENE-4100 (8.0.0), and ConstantScoreScorer empties its iterator under TOP_SCORES since LUCENE-8464 (8.0.0).

The trigger surface moved. In 8.10.0-9.11.x the default search(DrillDownQuery, Collector) and the 2-arg manager search with default hooks were affected, and drill-down counts were low too. From 9.12.0, #13568 and #13702 route hits and drill-down facets through MultiCollector(Manager), which forces a COMPLETE mode, so only the 3-arg search and the null hook remain. A minShouldMatch=2 base passes on 9.12.x and 10.0.0 and fails from 10.1.0 on for the tested data (not bisected).

Not fixed upstream: no 2026 commit on GitHub main touches DrillSidewaysScorer.java or DrillSidewaysQuery.java (checked 2026-11-03); the open PRs touching DrillSideways files (#16707, draft #15221, #16847) do not change createWeight or doQueryFirstScoring.

Versions and branches executed, with results
Version or commit	Verdict	Result
8.9.0	last clean	the history test passes: 75/75, drill-down 75
8.10.0	first: 8.10.0	default search(DrillDownQuery, Collector): sideways 11/10, drill-down 11; passes with only DrillSidewaysScorer reverted to before c7210e494e16
9.0.0	affected	fails the same way; passes with only DrillSidewaysScorer reverted to 61f8517000b^
8.11.4	default path	default search(DrillDownQuery, Collector): sideways 11/10, drill-down 11
9.11.1	default paths	default Collector path and 2-arg manager search with default hooks: 11/10, drill-down 11
9.12.0, 9.12.3	opt-in paths	default paths exact; 3-arg search with a filter base and the null hook: 11/10; this doc's test fails on 9.12.3 (assertions on)
10.0.0, 10.1.0, 10.2.0, 10.3.0, 10.4.0, 10.5.0, 10.5.2	affected	3-arg search, filter base: 11/10; minShouldMatch=2 base fails from 10.1.0; 2-arg null hook fails on 10.0.0 and 10.5.2
10.6.0	latest release	this doc's test fails with assertions on: 11/10 instead of 75/75
main 5adff78bb168	affected	this doc's test fails with assertions on and off (gradle and standalone runs); passes with the fix

Independent checks

The verifiers rated it real and the adjudication confirmed it (high confidence); verifiers wrote their own failing tests and a credibility audit added a multi-segment probe. A read-only cross-model review (OpenAI GPT-5.6) could not run code; its refute passes said not real on procedural grounds only. This doc's runs reproduced it on main, 10.6.0 and 9.12.3.

Checks and verdicts388 words

Most checks ran on main a9779ef3c193. Between it and 5adff78bb168, lucene/facet and the core search classes cited here are unchanged except TopFieldCollector.java and TopFieldCollectorManager.java (#16857), and the finder's two tests still fail the same way on 5adff78bb168 (22 and 11 instead of 100). No executing cross-model review was obtained.

Each independent check and its verdict
Check	Verdict	What it found
finder	reproduced	Two-test JUnit (filter base with `TopScoreDocCollectorManager(10, 10)`, `minShouldMatch=2` base with a relevance-sorted `TopFieldCollectorManager`) fails 2 of 2 on main (on, off; 20 of 20 with 10 iterations) and on 10.5.2; a 2-arg null-hook variant fails on main, 10.5.2 and 9.12.3; exhaustive and concurrent controls pass.
independent reproducer	real	Own 7-test JUnit fails 5 of 7 on main, 10.5.2, 10.0.0 and 9.12.3 (threshold 1000: 1001 instead of 2250; `scoreSubDocsAtOnce`: 11 instead of 1500); scoring `TermQuery` and SHOULD controls pass; an 8.x variant fails on 8.10.0 and 8.11.4 and passes on 8.9.0; a one-line `FilterScorable` probe makes 7 of 7 pass.
cross-model review (OpenAI GPT-5.6, read-only)	refute lens: not real (low, procedural); intended lens: real (medium)	The reviewer could not run code; its own notes call the refute verdict "not a substantive refutation", and its reading agrees with the root cause and scope. A separate run re-ran the finder's test: 2 of 2 fail.
verifier re-runs	real	Re-ran the finder's test with assertions off (2 of 2 fail) and the 2-arg variant (the null-hook test fails, the default-hook test passes).
devil's-advocate verifier	real (medium-high)	Own test with IndexSearcher's threshold 1000 on 60,000 docs fails 4 of 8 (1001 instead of 2000) while 4 controls pass (unlimited threshold, `MultiCollectorManager` with facets, concurrent mode, `TermQuery` base).
adjudication	confirmed (high)	Re-ran the independent reproducer's test (5 of 7 fail, on and off), the `FilterScorable` probe (7 of 7 pass) and the finder's test (2 of 2 fail).
credibility audit	real	Own multi-segment probe: an `IntPoint` range base with `TopScoreDocCollectorManager(10, 1000)` gives 1171 instead of 3000; scoring `TermQuery`, conjunction and MUST plus FILTER bases are exact.
this doc's runs	reproduced, fix validated	The one-test repro fails on main (gradle and standalone runs, on and off), 10.6.0 and 9.12.3; controls pass; fix A and fix A+B pass the repro and the facet tests; the score-mode change alone does not.

Diagram file

The diagram as a still image, generated with AI and checked against the code. It is under 10 MB, the limit for a GitHub attachment.

PNG 583 KB

Sources and provenance

Finding record, Lucene and upstream sources, repro files and diagram notes1,450 words

Finding record

LFV-789, snapshot of 2026-11-06

Registry title: Sequential DrillSideways lets a TOP_SCORES hit collector prune the shared base query, so drill-sideways facet counts silently miss matching documents
Area: facet / DrillSideways (sequential query-first scoring: score mode of the base weight and the scorable handed to the hit collector)
Priority: Medium, rank 36: Sequential DrillSideways silently undercounts sideways facets like LFV-358 but needs a caller-supplied finite-threshold top-N collector.
Impact class: silent wrong results
Impact note: Applications that run DrillSideways without an executor (the default sequential mode) and collect hits with a relevance-ranked top-N collector manager that has a finite total-hits threshold (TopScoreDocCollectorManager(n, t), including t = 1000, the threshold IndexSearcher#search(Query, int) uses, or TopFieldCollectorManager(Sort.RELEVANCE, n, t)), passed through the 3-arg search(DrillDownQuery, CollectorManager, List) or through the 2-arg search with createDrillDownFacetsCollectorManager() returning null (documented as valid), get drill-sideways facet counts that are silently too low (11 instead of 75; 1001 instead of 2250 at threshold 1000) whenever the base query's scorer can skip (filter-only or constant-score bases, minShouldMatch > 1 disjunctions) and the query-first strategy runs (base cost below a tenth of the drill-down cost, or scoreSubDocsAtOnce()); default codec, IndexWriterConfig and searcher, ordinary scale, -ea and -da alike; top hits stay correct, the concurrent DrillSideways and, since 9.12.0, the default 2-arg path are correct, while in 8.10.0-9.11.x the default search(DrillDownQuery, Collector) and the 2-arg manager search with default hooks undercounted both sideways and drill-down facets.
Summary: You run a sequential DrillSideways search (no executor) with a filter-style base query drilled down to color=red, and pass a TopScoreDocCollectorManager with a finite total-hits threshold (10 in the repro, or IndexSearcher's default 1000) as the hit collector through search(DrillDownQuery, CollectorManager, List). You expect the drill-sideways counts for color to include every document that matches the base query (75 in the repro), as the same search does with an exhaustive hit collector or with the concurrent DrillSideways. Instead the counts are silently too low (11, or 1001 instead of 2250 at threshold 1000): the base query is scored with the hit collector's TOP_SCORES mode and its scorer is handed to the hit collector, whose min-competitive-score updates make the shared base iteration skip documents that the sideways collectors then never see; the top hits are correct.
Introduced: 61f8517000b3af74c0b079e4a5fa81eb870b1c35 (LUCENE-10030, issue #11069, PR #217 by gtroitskiy, merged by gsmiller: lazy score() for near-miss documents in DrillSidewaysScorer.doQueryFirstScoring); branch_8x backport apache/lucene-solr#2542 = c7210e494e165b66673000df6ae43a32702800cf [exec: 8.10.0 passes with only DrillSidewaysScorer reverted to the parent of c7210e494e16 and fails with its own file; 9.0.0 passes with only DrillSidewaysScorer reverted to 61f8517000b^]. The change handed the hit collector ScoreCachingWrappingScorer(baseScorer), which forwards setMinCompetitiveScore to the base scorer, instead of the private ScoreAndDoc scorable whose setMinCompetitiveScore was the Scorable no-op. Older preconditions [read]: the base weight takes the caller's score mode since LUCENE-4100 (4fc5a872ded, 2017-12-07, 8.0.0), and ConstantScoreScorer empties its iterator under TOP_SCORES once minScore > score since LUCENE-8464 (b4449c73e4c, 2018-11-08, 8.0.0). Behaviour-preserving follow-ups: e44636c2808 (LUCENE-10036, #226) and f527eb3b12d (#12407). (2021-07-26 (main, 61f8517000b); 2021-08-01 (branch_8x backport c7210e494e16); first release 8.10.0 (release artifacts built 2021-09-21))
First affected version: 8.10.0 [exec: 8.9.0 passes; 8.10.0 fails, and passes again with only DrillSidewaysScorer reverted to its pre-backport version]
Affected versions: Every release from 8.10.0 through 10.6.0 (10.6.0 executed 2026-11-03 with -ea: the one-test repro gives 11/10 instead of 75/75) and main a9779ef3c193; not fixed upstream (upstream main f7bcc0abd816 has no lucene/facet commit after a9779ef3c193, and the fix-site lines are identical on branch_10x and branch_10_6). [exec] 8.10.0, 8.11.4, 9.0.0, 9.11.1, 9.12.0, 9.12.3, 10.0.0, 10.1.0, 10.2.0, 10.3.0, 10.4.0, 10.5.0, 10.5.2 and main (-ea and -da); [read] 8.10.1, 8.11.0-8.11.3, 9.1.0-9.11.0, 9.12.1-9.12.2, 10.2.1-10.2.2, 10.3.1-10.3.2 and 10.5.1. Trigger surface by release: in 8.10.0-9.11.x also the default search(DrillDownQuery, Collector) and the 2-arg manager search with default hooks (sideways and drill-down counts both low); from 9.12.0, #13568 and #13702 route hits and drill-down facets through MultiCollector(Manager), which forces COMPLETE, so only the 3-arg search and the 2-arg search with a null drill-down facets hook remain affected. A filter/constant-score base fails on every affected release; a minShouldMatch=2 disjunction base passes on 9.12.x and 10.0.0 and fails from 10.1.0 on for the tested data (not bisected). Not affected: 8.9.0 [exec], 8.0.0-8.8.x [read].
Status: documented (brief, checks and a per-finding doc exist)
Verification: Confirmed by AI-run checks: an independent repro, a cross-model review that could only read the code (its low-confidence “not real” is procedural), an intended-behaviour check, a prior-report search and a skeptical audit; not yet triaged by a person.
Found by: non-formal: code reading
Upstream and prior reports: related items only, no report of this defect (re-checked 2026-11-03; searched 2026-10-27: 199 queries from a scripted prior-report search in three runs plus manual searches over GitHub issues, PRs, commits and code, downstream repositories, the dev@, java-user@, issues@ and commits@ lists, the JIRA archive (LUCENE and SOLR) and the web, the full text and comments of #15220, #15221, #11069 and #217, and the changed files of 409 open apache/lucene PRs; 26 hits assessed, none reports this mechanism). Related, each about a different defect: apache/lucene#15220 (open, 2025-09, tflobbe) describes that in 8.10 a DrillSideways hit collector 'that would early terminate' also ended facet collection 'with incomplete counts', and asks for that as an opt-in feature on CollectionTerminatedException; the maintainer reply (gsmiller) says sibling collectors keep collecting; draft PR #15221 and #11040 (LUCENE-10001) carry the same request, and DrillSidewaysScorer's TODO from #12640 plus TestDrillSideways#testCollectionTerminated cover the explicit-termination case. None mentions setMinCompetitiveScore, TOP_SCORES, the base scorer or a finite hits threshold. Precedent: #9429 (LUCENE-8382: MultiCollector ignores setMinCompetitiveScore so that 'the other collector still sees all hits'). Introducing change: #11069 / #217 (LUCENE-10030), whose discussion covers only the cost of score(). Not fixed: no commit after a9779ef3c193 touches lucene/facet, and no open PR changes DrillSidewaysQuery.createWeight or doQueryFirstScoring (#16707 and draft #15221 touch DrillSideways.java only).
Lucene commit tested: main 5adff78bb168 (2026-11-03): the one-test repro through gradle in a clean scratch worktree, assertions on and off, stock (fails) and with fix A and with fix A+B (passes), plus TestDrillSideways, TestParallelDrillSideways and TestDrillDownQuery with A, the whole :lucene:facet:test and :lucene:facet:check -x test with A+B; the same test in standalone runs on the main jars (on, off) and, with assertions on, on the 10.6.0 and 9.12.3 jars (fails); two controls (unlimited threshold, concurrent mode: pass); the finder's two-test repro stock and with fixes A, B and A+B applied. Earlier runs by the finder, verifiers, judge and a credibility audit on main a9779ef3c193, a re-run on main e1faeba08bde, and a history sweep of 32 runs from 8.9.0 to main with one-file reverts on 8.10.0 and 9.0.0. All runs used OpenJDK 25.0.3 on Linux aarch64.

Lucene and upstream sources

Finding record: lucene/facet/src/java/org/apache/lucene/facet/DrillSidewaysQuery.java:126-135 (at a9779ef3c193) (createWeight: base weight built with the caller's scoreMode at :129; drill-down weights with ScoreMode.COMPLETE_NO_SCORES at :133-134)
lucene/facet/src/java/org/apache/lucene/facet/DrillSidewaysScorer.java:130 (at a9779ef3c193) (query-first when scoreSubDocsAtOnce or baseQueryCost < drillDownCost / 10), :197-200 (doQueryFirstScoring: ScoreCachingWrappingScorer.wrap(collector); setScorer(collector, baseScorer)), :149-151 (TODO on CollectionTerminatedException from #12640)
lucene/facet/src/java/org/apache/lucene/facet/DrillSideways.java:57-64 (at a9779ef3c193) (the standard and concurrent implementations are described as differing in speed and duplicate work, with no mention of different results), :69 (@lucene.experimental), :133-139 (createDrillDownFacetsCollectorManager: 'Returning null is valid'), :271-280 (DrillSidewaysResult: sideways collectors expose 'the hits that match on all DrillDownQuery dimensions, but treating their corresponding sideways dimension as optional'), :398-405 (3-arg search: 'can be used with any CollectorManagers'), :467 (searchSequentially: searcher.search(dsq, drillDownCollectorManager))
lucene/core/src/java/org/apache/lucene/search/ConstantScoreScorer.java:150 (at a9779ef3c193) (TOP_SCORES and minScore > score: the iterator becomes empty); MultiCollector.java:197-200 (setMinCompetitiveScore ignored so that 'the other collector still sees all hits')
The finding in 30 seconds: lucene/facet/src/java/org/apache/lucene/facet/DrillSidewaysQuery.java:126-135 (at 5adff78bb168) (:129, :133-134)
lucene/facet/src/java/org/apache/lucene/facet/DrillSidewaysScorer.java:130, (at 5adff78bb168) :199-200, :336, :548, :697-735, :746-751, :753-764
lucene/facet/src/java/org/apache/lucene/facet/DrillSideways.java:50-52, (at 5adff78bb168) :467
lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollector.java:58-60, (at 5adff78bb168) :155-162; IndexSearcher.java:100, :599; ScoreCachingWrappingScorer.java:93-96; ConstantScoreScorer.java:149-153 (:150); TopFieldCollector.java:337-343; WANDScorer.java:258-267; Scorable.java:45-52
Who is affected: lucene/facet/src/java/org/apache/lucene/facet/DrillSideways.java:133-135, (at 5adff78bb168) :325-339 (:336, :338), :399-400, :402-404; DrillSidewaysScorer.java:714-719, :730-735
lucene/core/src/java/org/apache/lucene/search/MultiCollector.java:104-121; (at 5adff78bb168) TopScoreDocCollector.java:116-119; TopFieldCollector.java:337-343; ConstantScoreScorer.java:150; IndexSearcher.java:84
Why it looks unintended: lucene/facet/src/java/org/apache/lucene/facet/DrillSideways.java:50-52, (at 5adff78bb168) :57-62, :69, :133-135, :200-201, :232-233, :272-274, :399-400, :402-404
lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollectorManager.java:50-54; (at 5adff78bb168) MultiCollector.java:104-121, :195-202; Scorable.java:45-52
lucene/facet/src/java/org/apache/lucene/facet/DrillSidewaysScorer.java:149-151; (at 5adff78bb168) lucene/facet/src/test/org/apache/lucene/facet/TestDrillSideways.java:258 (testCollectionTerminated)
https://github.com/apache/lucene/issues/9429; https://github.com/apache/lucene/issues/11069; https://github.com/apache/lucene/pull/217
Suggested fix: lucene/core/src/java/org/apache/lucene/search/FilterScorable.java:27-28; (at 5adff78bb168) MultiCollector.java:104-121, :195-202
lucene/facet/src/java/org/apache/lucene/facet/DrillSidewaysScorer.java:746-751, (at 5adff78bb168) :760-763
https://github.com/apache/lucene/issues/11128; https://github.com/apache/lucene/issues/11076
Independent checks: Lucene main 5adff78bb168 git diff --stat a9779ef3c193 HEAD -- lucene/facet and the core search classes cited (only TopFieldCollector.java and TopFieldCollectorManager.java changed, #16857 70a428b5ba9)
Diagrams: lucene/facet/src/java/org/apache/lucene/facet/DrillSidewaysScorer.java:199-200, (at 5adff78bb168) :746-751; lucene/core/src/java/org/apache/lucene/search/TopScoreDocCollector.java:155-162; (at 5adff78bb168) ConstantScoreScorer.java:149-153

Other statements trace to the hunt's working files (models, logs, reviews and briefs), which are not published.

Repro files

TestDrillSidewaysTopScoresHitCollector.java (one test, lucene/facet): repro/LFV-789/TestDrillSidewaysTopScoresHitCollector.java (91 lines)

About the diagrams

A step-by-step picture of the mechanism, made in parallel with this doc from the code at 5adff78bb168 and the executed runs; the PNG is the summary still.

assets/LFV-789/bug.png (583 KB)

Built 2026-11-06 from the hunt's record of this finding and its doc file. Lucene main under test in this hunt: ea57360912ed. Every listed finding was re-run on Lucene main e1faeba08bde on 2026-11-02.

The finding in 30 seconds

Two things to know first

The finding, step by step

Who is affected

The code (main 5adff78bb168)

Minimal repro

Environment

Observed output

Why it looks unintended

Suggested fix

FilterScorable for the collectors (the minimal fix) (validated by the checks) (recommended)

Optional extra: also build the base weight with an exhaustive score mode (validated by the checks)

Documentation only: require an exhaustive hit collector

Downstream note

How it was found

Prior reports, history and checks

Prior reports

History

Independent checks

Diagram file

Sources and provenance

Finding record

Lucene and upstream sources

Repro files

About the diagrams

The code (main `5adff78bb168`)