PositiveScoresOnlyCollector gets the hits wrong when it wraps a collector that needs no scores

PositiveScoresOnlyCollector wraps another collector and is documented to make "sure only documents with scores > 0 are collected"; the Collector overview says it "wraps any other Collector". To check a score it reads scorer.score() itself, but it extends FilterCollector and inherits two hooks meant for pure delegators: scoreMode() reports the wrapped collector's needs, and setWeight() forwards the Weight. Around a collector that needs no scores, both go wrong. The search builds scorers without scores, so the > 0 check reads placeholder values: 0 for a TermQuery, so a sort-by-field TopFieldCollector gets 0 of the 6 hits of g:y; a constant 1 for a FunctionScoreQuery over MatchAllDocsQuery, so documents whose score is 0 get through (docs 0-5 instead of 1, 3, 5). And a wrapped TotalHitCountCollector counts whole segments with Weight#count and never runs the check: 12 instead of 0 for (f:x)^0, 12 instead of 6 for (g:y)^2 OR (f:x)^0. No exception is raised, with assertions on or off, with the default codec and settings and 12 documents. The score-mode half dates from LUCENE-6218 (first released in 5.1.0), the count half from LUCENE-10620 / #964 (9.3.0); neither edited the class. Overriding both hooks in PositiveScoresOnlyCollector fixes every repro, and the search-package and queries-module test suites pass with it. Wrapping a scoring collector such as TopScoreDocCollector, the documented use, has always worked, and the class has no known in-tree or major downstream user.

• ScoreMode: what a collector needs from the scorer. COMPLETE means "visit all matches and get their score"; COMPLETE_NO_SCORES means "scores won't be available"; TOP_DOCS (a sort by a field with the default hit-count threshold) also needs no scores. IndexSearcher builds the query's Weight and scorers for the collector's mode.
• FilterCollector: a base class for collector delegators. It forwards scoreMode() and setWeight() to the wrapped collector.
• Weight#count: a per-segment shortcut that returns the number of matches without iterating (-1 when unknown). TotalHitCountCollector uses it when setWeight gave it the Weight, and then skips the segment.

PositiveScoresOnlyCollector (PositiveScoresOnlyCollector.java:26-52 at a9779ef3c193) overrides only getLeafCollector, whose leaf collector does if (scorer.score() > 0) in.collect(doc) (:47). It inherits:

1. scoreMode() = in.scoreMode() (FilterCollector.java:51-54). Around a collector that needs no scores, IndexSearcher creates the Weight without scores and rewrites the query as ConstantScore(query), whose weight is then the plain inner weight. A TermQuery without scores uses a dummy SimScorer that returns 0 and a constant-0 bulk scorer, so every hit fails > 0. A FunctionScoreQuery without scores returns its inner weight; MatchAllDocsQuery's weight scores 1, so every hit passes, including those whose function value is 0.
2. setWeight() = in.setWeight(weight) (FilterCollector.java:41-44). A wrapped TotalHitCountCollector asks weight.count(context), adds it and throws CollectionTerminatedException, so the wrapper's leaf collector never runs for that segment. With deletions Weight#count returns -1, collect() runs, and half 1 takes over.

Each half is an independent cause: fixing only scoreMode() still leaves 12 of 24 count searches of the independent reproducer wrong, fixing only setWeight() 16 of 24 in each of its tests; fixing both, all pass.

Over a bounded-exhaustive grid (2 segment layouts, 2 deletion states, 8 queries, 3 search paths, 10 wrapped collectors covering 4 of the 5 ScoreMode values, all but TOP_DOCS_WITH_SCORES), 384 of 960 checks are wrong, every one with one of the 5 wrapped collectors that need no scores (384 of their 480 checks; the 5 scoring ones are right in all 480), and 0 with the fix.

• Who is exposed. Code that wraps a sort-by-field TopFieldCollector (the collector behind IndexSearcher#search(Query, int, Sort)), a TotalHitCountCollector, a custom COMPLETE_NO_SCORES collector or a MultiCollector of such collectors. The count half also bites when the search does compute scores: MultiCollector(PositiveScoresOnlyCollector(TotalHitCountCollector), TopScoreDocCollector) has the right score mode, but the forwarded Weight still lets the counter skip the check (6 of 8 probe cases wrong from 9.3.0).
• Default vs opt-in. The class is opt-in; everything else is default: codec, IndexWriterConfig, IndexSearcher, both search paths. A plain TermQuery triggers the sort case; a zero boost (legal) or a function score gives zero-score hits.
• Effect. Wrong hits or counts in both directions: missing hits that score above 0, extra hits that score 0, counts too high (zero-score matches counted) or too low (with deletions). No exception.
• How likely. Low. Outside its own test nothing in Lucene uses the class, no use was found in Elasticsearch, OpenSearch or Solr, the one user code found upstream (#13499) wraps a scoring collector, which works, and since 8.0 ("Queries are now required to produce positive scores.", LUCENE-7996) the class only removes hits with a score of exactly 0.

# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 2 test(s) run, 2 failed
    => java.lang.AssertionError: count (expected: none, the one match scores 0) expected:<0> but was:<1>
       org.junit.Assert.fail(Assert.java:89)
       org.junit.Assert.failNotEquals(Assert.java:835)
       org.junit.Assert.assertEquals(Assert.java:647)
       org.apache.lucene.search.TestPositiveScoresOnlyCollectorWrapsNoScoreCollector.testWrappedTotalHitCountCollector(TestPositiveScoresOnlyCollectorWrapsNoScoreCollector.java:59)
  JUnit Vintage:TestPositiveScoresOnlyCollectorWrapsNoScoreCollector:testWrappedSortByFieldCollector
    MethodSource [className = 'org.apache.lucene.search.TestPositiveScoresOnlyCollectorWrapsNoScoreCollector', methodName = 'testWrappedSortByFieldCollector', methodParameterTypes = '']
    => java.lang.AssertionError: hits (expected: the one match, its score is > 0) expected:<1> but was:<0>
       org.junit.Assert.fail(Assert.java:89)
       org.junit.Assert.failNotEquals(Assert.java:835)
       org.junit.Assert.assertEquals(Assert.java:647)
       org.apache.lucene.search.TestPositiveScoresOnlyCollectorWrapsNoScoreCollector.testWrappedSortByFieldCollector(TestPositiveScoresOnlyCollectorWrapsNoScoreCollector.java:52)

• Javadoc. Class: "A Collector implementation which wraps another Collector and makes sure only documents with scores > 0 are collected." Collector overview: "PositiveScoresOnlyCollector wraps any other Collector and prevents collection of hits whose score is <= 0.0". Collector#scoreMode: "Indicates what features are required from the scorer." ScoreMode.COMPLETE_NO_SCORES: "scores won't be available". No CHANGES or MIGRATE text restricts the wrapped collector.
• In-tree precedent. CachingCollector.ScoreCachingCollector.scoreMode() returns COMPLETE: "Ensure the scores are collected so they can be replayed, even if the wrapped collector doesn't need them." It came as a bug fix: LUCENE-6912 (2015-12-04, first released in 5.5.0) added it, and its commit message says "In core, CachingCollector should have always returned true when scores are cached (small bug)."
• The test framework trips too, through a neighbouring rule. Under AssertingIndexSearcher with assertions on, the assertion "Scorable.score() must not be called inside collect(DocIdStream) or collectRange() ..." fires inside PositiveScoresOnlyCollector.collect for both no-score wrapped collectors the finder's probe tried (3 seeds), and never with the fix. That assertion is about batch collection, not about whether scores were requested: the no-score bulk scorer hands over a DocIdStream, which the wrapper's inherited default collect(DocIdStream) turns into collect(int) calls that read score().
• Tests. TestPositiveScoresOnlyCollector.testNegativeScores drives a TopScoreDocCollector by hand with a fake scorer; it never runs a search, never wraps a no-score collector and never calls setWeight.
• History. The class (LUCENE-1575, 2.9) predates both hooks; its CHANGES usage example wraps a TopScoreDocCollector. LUCENE-6218 (2015-02) added needsScores forwarding to FilterCollector; in CachingCollector it only gave the placeholder collector inside CachingCollector.create a needsScores() of true, and it did not touch this class. CachingCollector's own override came ten months later with LUCENE-6912, as the fix of "a small bug"; #964 (2022) added setWeight forwarding and never discusses wrappers that filter collect() (all comments and reviews read).

The strongest case for "intended" (the intended-behaviour checker): for the count half, #964 met this exact interaction in its own tests. A test wrapper that filters collect() around a TotalHitCountCollector stopped working, and the author moved the test to a test-only counter: "I fixed some tests that were assuming that TotalHitCountCollector would naively iterate over matches". TotalHitCountCollector's javadoc documents the segment skipping. For the score half: the class was built to wrap a scoring collector, and LUCENE-6329 only says score() must not throw when scores were not requested, not that it is meaningful. And Collector and FilterCollector are @lucene.experimental.

Why it fails: the #964 change concerned a private test class and never mentions PositiveScoresOnlyCollector, whose public javadoc the forwarding breaks; it does not touch the score half at all, where hits that really score above 0 are dropped. The component that needs scores is Lucene's own wrapper; in #15239, about a user's collector that wraps a TopScoreDocCollector with a different score mode, a maintainer wrote "It is the responsibility of the collector to prevent such calls, so this is not a bug": the wrapper was at fault, and here the wrapper is Lucene's own. Lucene's sibling wrapper overrides scoreMode for this reason (LUCENE-6912 called the inherited forwarding "a small bug"), and no won't-fix or by-design statement exists. Verdict: not intended (score half: high confidence; count half: medium, because of the #964 precedent). It is a design inference: no javadoc says a FilterCollector subclass must override these hooks, and for the count half documenting the restriction would also be a defensible maintainer choice.

Both shapes of the setWeight change were validated on the module test suites; they differ in what the wrapped collector can still do with the Weight.

scoreMode() keeps the wrapped mode when it needs scores and maps TOP_DOCS and COMPLETE_NO_SCORES to COMPLETE. setWeight() wraps the Weight in a FilterWeight whose count() returns -1, so the wrapped collector can no longer count a segment without collect(), but keeps matches, explain and the scorers. This is the finder's fix, in the shape an independent reviewer from another model family proposed.

@Override
public ScoreMode scoreMode() {
  // This collector reads scores itself, even if the wrapped collector does not need them.
  ScoreMode scoreMode = in.scoreMode();
  return scoreMode.needsScores() ? scoreMode : ScoreMode.COMPLETE;
}

@Override
public void setWeight(Weight weight) {
  // The wrapped collector must not count matches with Weight#count and skip collect(): that would
  // also count matches whose score is <= 0.
  in.setWeight(
      weight == null
          ? null
          : new FilterWeight(weight) {
            @Override
            public int count(LeafReaderContext context) {
              return -1;
            }
          });
}

Trade-offs

• A wrapped TopScoreDocCollector keeps TOP_SCORES pruning. For a wrapped sort-by-field or counting collector the Weight is now built for COMPLETE: scores are computed, which the wrapper needs anyway, and the search is exhaustive, so a query loses any shortcut it takes for a non-exhaustive mode (not benchmarked). By code reading no competitive-iterator skipping is lost: the wrapper's leaf collector does not pass one on today.
• setWeight() hides Weight#count from every wrapped collector, scoring or not; one that counts with it would count in collect() instead. By code reading the only in-tree collectors that call Weight#count are TotalHitCountCollector and the sandbox HistogramCollector, and neither needs scores.
• Targeted: it blocks only Weight#count and keeps every other Weight method (matches, explain, scorerSupplier, getQuery) for per-document users (Lucene's monitor module calls weight.matches(ctx, doc) from its own collectors).
• FilterWeight is @lucene.internal, which is fine inside core.
• Gap, by code reading only: getQuery() still returns the query, and the sandbox HistogramCollector uses it to bulk-count a PointRangeQuery without collect(). That matters for a point range whose scores can be 0 (a zero boost, or a FunctionScoreQuery around it whose values can be 0: HistogramCollector unwraps both), wrapped through a custom manager (the collector is package-private). Not executed.
• Not benchmarked; only this class changes.

What the checks validated

Scratch worktree at a9779ef3c193, gradle, seed DEADBEEF0353: :lucene:core, every test class of org.apache.lucene.search with the 3 repro classes: 217 classes, 2,340 tests, 0 failures, with assertions on and off (TestPositiveScoresOnlyCollector, TestTotalHitCountCollector, TestMultiCollector, TestTopFieldCollector, TestCachingCollector, TestIndexSearcher among them); the whole :lucene:queries suite with the fail-open test: 52 classes, 513 tests, 0 failures, on and off. :lucene:core:check -x test and :lucene:queries:check -x test with the fix and both tests: BUILD SUCCESSFUL. The repro runner: the primary test passes 3 of 3 with the fix; the 960-check harness finds 0 wrong.

The same scoreMode() override, and an empty setWeight (+13 lines). This is what downstream engines did for their own min-score wrappers: OpenSearch gave its MinimumScoreCollector (a FilterCollector subclass) a no-op setWeight (PR #16366), and Elasticsearch withholds setWeight under min_score (#97030).

@Override
public void setWeight(Weight weight) {
  // Do not pass the weight on: the wrapped collector could count matches with Weight#count and
  // skip collect(), which would also count matches whose score is <= 0.
}

Trade-offs

Simpler, and closes every Weight-based shortcut, including the HistogramCollector one. Cost: the wrapped collector never gets the setWeight callback (it is skipped, not called with null), so a per-document user of the Weight has none, and a collector that requires the callback breaks: by code reading, the test framework's AssertingCollector asserts that it was called, so one wrapped inside the class would fail with assertions on (not executed).

What the checks validated

Gradle, assertions on: the same selection, 217 classes, 2,340 tests and 52 classes, 513 tests, 0 failures. The finder also ran it on the repro, the upstream test (3 seeds), the 960-check harness and the asserting probe (3 seeds), while it was the finder's only fix: all pass. Not run with assertions off or through the static checks.

Ask for scores but keep forwarding the Weight.

Trade-offs

Not enough: the counter still skips the check through Weight#count (12 of 24 count searches still wrong in the independent reproducer's shadow run).

scoreMode() returns COMPLETE whatever the wrapped collector needs.

Trade-offs

Correct together with a setWeight change, but drops TOP_SCORES pruning for the documented TopScoreDocCollector use. Not run.

Say in the javadoc that only collectors that need scores may be wrapped, and never a TotalHitCountCollector.

Trade-offs

Contradicts "wraps any other Collector" and keeps a public class that silently fails for natural uses. Not run.

Subclass PositiveScoresOnlyCollector with the two overrides of the recommended fix, or filter by score in the application's own collector.

Trade-offs

Every application has to know about it.

Method: semi-formal (a model-written probe, then bounded-exhaustive differential testing of the real core jar, guided by mutation analysis).

• First signal (2026-10-28). A planning round asked an independent model from another family (OpenAI GPT-5.6) for testing-research leads; one lead's probe ran on the stock main jar: a wrapped collector that needs no scores collected 0 of 6 positive hits, and a wrapped TotalHitCountCollector reported 12 instead of 0 or 6.
• Hunt. The finder checked the wrapper against an independent reference, the matches whose COMPLETE score is above 0, over 2 segment layouts, 2 deletion states, 8 queries, 3 search paths and 10 wrapped collectors covering 4 of the 5 ScoreMode values (all but TOP_DOCS_WITH_SCORES): 384 of 960 checks wrong, every one with a wrapped collector that needs no scores. Mutants of the wrapper that override only scoreMode(), only setWeight(), or both separated the two causes; a mutant of the reference (>= 0) checked that the oracle was not vacuous.
• A class-file inventory of the 8 Collector wrapper classes in the non-sandbox jars flagged the same pattern (reads scores, inherits both hooks) only in CachingCollector's classes, a separate hunt finding (LFV-161), partly known upstream.
• The finder then wrote a public-API JUnit test, which failed on main and on every release run from 8.0.0.