Single-valued SortedSetDocValues: nextOrd() after intoBitSet returns an earlier document's ordinal, so DocValuesRangeIterator.forOrdinalRange can return extra hits

The finding in 30 seconds

The view caches the current document's ordinal and refreshes it in nextDoc, advance and advanceExact. Its intoBitSet, added in 10.3.0, only forwards the call to the inner iterator, so the iterator moves while the cached ordinal stays behind.

Background: ordinals, the single-valued view and skip blocks

A SortedSetDocValuesField stores each document's values as ordinals, positions in the field's sorted list of distinct values ("a" = 0, "b" = 1). You position on a document with nextDoc() or advanceExact(doc), then call nextOrd() up to docValueCount() times.
When every document of a segment has at most one value, the default codec stores the field as single-valued and hands out a thin multi-valued view over it. Lucene's core queries unwrap it with DocValues.unwrapSingleton; other code, including the sandbox SortedSetDocValuesMultiRangeQuery, may use it as it is.
intoBitSet(upTo, bits, offset) is a bulk shortcut: mark the remaining documents below upTo and stop on the first document at or after upTo, exactly as a nextDoc() loop would.
With a skip index (SortedSetDocValuesField.indexedField), DocValuesRangeIterator sorts skip blocks (by default every block but a segment's last holds at least 4,096 documents with a value) into classes: YES_IF_PRESENT when every value present is in the range (mark the documents that have one, in bulk), MAYBE when each document's ordinal must be tested.

Summary card in four panels. 1, what you do: two documents "a" (ordinal 0) and "b" (ordinal 1); nextDoc(), intoBitSet(1, new FixedBitSet(1), 0), nextOrd(): expected 1, actual 0. 2, inside the view after intoBitSet: the wrapped iterator is on doc 1 but the cached ord is still 0, doc 0's ordinal; nextDoc() and advance() refresh ord, intoBitSet only delegates; in the range example the view sits on doc 8192 ("a") with ord 1 ("b"); the fix refreshes ord unless the iterator is exhausted. 3, expected vs actual on main with assertions on and off: nextOrd() 1 vs 0; forOrdinalRange [b, b] over 16,384 documents marks 8,193 instead of 8,192, the extra one being doc 8192 whose value is "a"; the custom query as a FILTER next to FieldExistsQuery counts 8,193 instead of 8,192; the unwrapped SortedDocValues control and the fix give 8,192. 4, why it is a bug and who is affected: the intoBitSet contract and nextOrd() for the current document; Lucene's core queries unwrap the view; affected is custom code that passes the view of a skip-indexed field to forOrdinalRange or reads nextOrd() after intoBitSet, and the sandbox SortedSetDocValuesMultiRangeQuery when its ranges collapse to one; extra hits only, at most one per YES_IF_PRESENT block (by code reading). — A summary card: the repro's calls, the view's stale cached ordinal after `intoBitSet`, expected versus actual (`nextOrd()` 1 vs 0; 8,192 vs 8,193 documents for [b, b]) and who is affected. Select the still to open it at full size. Generated with AI, checked against the code.

The finding, step by step

Index two documents with a SortedSetDocValuesField f: doc 0 = "a" (ordinal 0), doc 1 = "b" (ordinal 1). Each has one value, so the default codec stores f as single-valued and DocValues.getSortedSet hands out a thin view that remembers the current document's ordinal.
Call nextDoc() (doc 0), then intoBitSet(1, bits, 0). Its javadoc says it behaves exactly like a nextDoc() loop: it marks doc 0 and leaves the iterator on doc 1.
You expect nextOrd() to return 1, doc 1's "b". You get 0, doc 0's "a", with no error: the view's intoBitSet moved the inner iterator but did not update the remembered ordinal.
The consequence: forOrdinalRange loads a skip block with intoBitSet, then tests the next block's first document with nextOrd(). On 16,384 skip-indexed documents the range [b, b] also matches doc 8192, whose value is "a": 8,193 hits instead of 8,192 in a count of a FILTER conjunction or a field-sorted filtered search.

Who is affected

Only custom code: code that reads nextOrd() after intoBitSet on a single-valued segment, or that hands this view of a skip-indexed field to the experimental forOrdinalRange (or uses the sandbox multi-range query) and evaluates it in bulk: counts, field-sorted or no-scores filters. Core queries unwrap the view first; top-scores searches were correct.

Mechanism, terms and numbers969 words

When every document of a segment has at most one value for a SortedSetDocValuesField, the default codec stores the field as single-valued and hands out SingletonSortedSetDocValues, a multi-valued view that caches the current document's ordinal. nextDoc, advance and advanceExact refresh the cache; the intoBitSet override added by #14618 (first released in 10.3.0) only forwards to the wrapped SortedDocValues, so after it the view reports the new document while nextOrd() still returns an earlier document's ordinal. That breaks the DocIdSetIterator#intoBitSet contract ("should behave exactly as if implemented as" the nextDoc() loop) and nextOrd()'s ("the next ordinal for the current document"). Two documents suffice: after nextDoc() and intoBitSet(1, ...), nextOrd() returns 0 instead of 1, with assertions on and off. Since 10.5.0, DocValuesRangeIterator.forOrdinalRange(SortedSetDocValues, ...) turns this into extra hits: it bulk-loads a YES_IF_PRESENT skip block with intoBitSet, then tests the next MAYBE block's first document with the stale ordinal. Core queries unwrap the view first and are not affected; exposed is code that reads ordinals after intoBitSet, or passes the reader's view of a skip-indexed field to the experimental factory and evaluates it in bulk (counts, field-sorted or no-scores filters), plus the sandbox multi-range query. #16714 fixed the identical omission in the SortedSetSelector wrappers in 2026 and did not touch this class. A 4-line fix (refresh the ordinal after in.intoBitSet) passes the test and the doc-values, range-iterator and codec suites.

The mechanism (main `645d4fe14a20`)

The view. Lucene90DocValuesProducer#getSortedSet returns DocValues.singleton(getSorted(entry.singleValueEntry)) for a field that is single-valued in the segment (Lucene90DocValuesProducer.java:2282-2286). DocValues.getSortedSet passes the reader's view through, or wraps a SORTED field the same way (DocValues.java:325-336); DocValues.unwrapSingleton returns the inner SortedDocValues (:177-183).
The cache. SingletonSortedSetDocValues has private long ord (SingletonSortedSetDocValues.java:31). nextOrd() returns it (:54-57), docValueCount() is 1 (:59-62), and nextDoc(), advance() and advanceExact() set ord = in.ordValue() on a hit (:64-89).
The override (#14618, :91-94):

   @Override
   public void intoBitSet(int upTo, FixedBitSet bitSet, int offset) throws IOException {
     in.intoBitSet(upTo, bitSet, offset);
   }

docID() is in.docID() (:49-52), so afterwards the view reports the new document while ord keeps the old ordinal.

The contract. DocIdSetIterator#intoBitSet: "This should behave exactly as if implemented as below, which is the default implementation: for (int doc = docID(); doc < upTo; doc = nextDoc()) bitSet.set(doc - offset);" (DocIdSetIterator.java:186-210, @lucene.internal). Through that loop the view's own nextDoc() refreshes ord. SortedSetDocValues#nextOrd: "Returns the next ordinal for the current document" (SortedSetDocValues.java:35-36).
The consumer. forOrdinalRange(SortedSetDocValues, skipper, min, max) builds a predicate on values.nextOrd() and, with a skipper, a BulkOrdinalRangeIterator (DocValuesRangeIterator.java:109-133). Its intoBitSet walks skip blocks (:349-371): a YES_IF_PRESENT block is loaded with disi.intoBitSet(blockEnd, ...) (:357-366), which leaves disi on the next block's first document; the following MAYBE block advances only if disi.docID() < blockStart (it is not) and evaluates the predicate at disi.docID() (:537-548), reading the stale ord. applyMask has the same YES_IF_PRESENT branch (:404-408).
Block classes. All values of a block in range and its documents contiguous gives YES; in range but with gaps gives YES_IF_PRESENT; otherwise MAYBE (SkipBlockRangeIterator.java:90-98). A skip block closes once it holds at least 4,096 documents with a value (the default, Lucene90DocValuesFormat.java:226); it grows past that only while the run stays single-valued, constant and without gaps (Lucene90DocValuesConsumer.java:250-262).

The consequence, block by block

16,384 documents with SortedSetDocValuesField.indexedField("f", ...): docs 0-8191 have "b" on even docs only, docs 8192-16383 have "a" on even and "b" on odd docs. Block 0 is docs 0-8190 with 4,096 values, all "b", with gaps: YES_IF_PRESENT for [b, b]. Block 1 starts at doc 8192 ("a"): MAYBE. For block 0 the iterator calls disi.advance(0), which sets ord at doc 0 (DocValuesRangeIterator.java:362-364, SingletonSortedSetDocValues.java:74-80); the following disi.intoBitSet(8191, ...) (:365) never writes ord. So after block 0 the view sits on doc 8192 with the ordinal it cached at doc 0 ("b"), and the predicate accepts doc 8192 (a fact-check probe with doc 0 = "b" and docs 2-8190 = "c" gets ord("b") there, not ord("c")). Expected 8,192 matches, got 8,193 (an independent verifier asserted this block layout through the skipper).

When it happens and when not

The root cause needs only a single-valued segment and a caller that reads nextOrd() after a valid SortedSetDocValues#intoBitSet call (the iterator positioned and offset <= docID(), as DocIdSetIterator.java:196-197 requires; the repro meets both).
The extra hit needs all of: a skip index on the field (without one, forOrdinalRange returns a per-document iterator, DocValuesRangeIterator.java:129-130); the reader's view passed without unwrapping to forOrdinalRange, or to the TermsEnum overload of forOrdinalSet when the matching terms form a contiguous ordinal range (:215-219, :246-247); bulk evaluation through intoBitSet or applyMask, as DenseConjunctionBulkScorer and ConstantScoreBulkScorer do for counts and for field-sorted or no-scores conjunctions (BooleanScorerSupplier.java:440-446); and a YES_IF_PRESENT block directly followed by a MAYBE block whose first valued document is out of range.
Not affected: the core SortedSetDocValuesField.newSlowExactQuery in the same search shapes, top-scores searches with the custom filter, per-document matches() over the same iterator, the same layout made multi-valued (the codec's multi-valued iterators reset their state in intoBitSet, Lucene90DocValuesProducer.java:2373-2380 and :2433-2436), all executed; and by code reading a forOrdinalSet call with a non-contiguous set or through the LongBitSet overload, which always passes contiguous=false (DocValuesRangeIterator.java:230-239; per-document iterator, :249-265), and a field that every document of a block carries (such a block is YES or MAYBE, never YES_IF_PRESENT, SkipBlockRangeIterator.java:91-97).
Direction: by code reading, the stale ordinal comes from an in-range document of the preceding YES_IF_PRESENT block, so the range path only adds documents, at most one per such block boundary. Not checked exhaustively.

Exposure and likelihood384 words

No core query. SortedSetDocValuesRangeQuery unwraps the view before building the iterator (SortedSetDocValuesRangeQuery.java:122-123, :159-164), and so does DocValuesRewriteMethod (DocValuesRewriteMethod.java:165-174); the core newSlowExactQuery counted correctly in the same search shapes.
Who is exposed. Code that reads nextOrd() after SortedSetDocValues#intoBitSet on a single-valued segment; code that passes the reader's view of a skip-indexed field to DocValuesRangeIterator.forOrdinalRange (or to the TermsEnum overload of forOrdinalSet when the matching terms form a contiguous ordinal range) and lets it run in bulk (the intended-behaviour check built such a filter from public classes that are not @lucene.internal, its own Query, Weight, ScorerSupplier and Scorer; the independent reproduction's filter uses ConstantScoreWeight and ConstantScoreScorer (public) and Weight.DefaultScorerSupplier (a protected nested class of Weight, usable from a Weight subclass), all three @lucene.internal; both miscount in a FILTER conjunction); and the sandbox SortedSetDocValuesMultiRangeQuery on a skip-indexed field when its ranges collapse to one ordinal range, which miscounts in a FILTER conjunction on main and 10.5.2.
Effect. Silent extra hits: IndexSearcher#count of a FILTER conjunction (the custom filter plus a FILTER on another field) 8,193 instead of 8,192 (counted alone, an independent verifier's filter gives the correct 8,192), and a field-sorted MUST+FILTER search or a no-scores collector returns doc 8192, whose value "a" is outside the filter's [b, b] (main with assertions on and off, 10.5.2). A verifier's own layout gave 4,151 instead of 4,150. Top-scores searches were correct in every shape tried.
Default vs opt-in. The root cause is on the default codec, for every single-valued segment. The extra hits need the opt-in skip index and the @lucene.experimental factory called without unwrapping.
Scale. By code reading, at most one extra document per YES_IF_PRESENT block directly followed by a MAYBE block (every block but a segment's last holds at least 4,096 documents with a value); the total depends on how many such boundaries a segment has, so there is no small fixed bound per segment. The data shape is ordinary: a sparse field (some documents lack it) where a block whose present values are all in range is followed by a mixed block, for example a low-cardinality status field.
Not affected. Data on disk, multi-valued segments, and deleted documents (live docs are applied to the window separately, by code reading: DenseConjunctionBulkScorer.java:281-283, ConstantScoreBulkScorer.java:141-142).

Minimal repro

One JUnit test in lucene/core, public classes and methods only, default codec: two documents, nextDoc(), intoBitSet(1, ...), then nextOrd(). It fails on main with assertions on and off; the same two-document check fails on every release since 10.3.0 and passes on 10.2.2.

TestSingletonSortedSetIntoBitSetStaleOrd.java (lucene/core test, one method)49 lines

/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.apache.lucene.index;

import org.apache.lucene.document.Document;
import org.apache.lucene.document.SortedSetDocValuesField;
import org.apache.lucene.store.ByteBuffersDirectory;
import org.apache.lucene.store.Directory;
import org.apache.lucene.tests.util.LuceneTestCase;
import org.apache.lucene.tests.util.TestUtil;
import org.apache.lucene.util.BytesRef;
import org.apache.lucene.util.FixedBitSet;

public class TestSingletonSortedSetIntoBitSetStaleOrd extends LuceneTestCase {

  public void testNextOrdAfterIntoBitSet() throws Exception {
    try (Directory dir = new ByteBuffersDirectory()) {
      IndexWriterConfig iwc = new IndexWriterConfig().setCodec(TestUtil.getDefaultCodec());
      try (IndexWriter w = new IndexWriter(dir, iwc)) {
        for (String value : new String[] {"a", "b"}) {
          Document doc = new Document();
          doc.add(new SortedSetDocValuesField("f", new BytesRef(value)));
          w.addDocument(doc);
        }
      }
      try (DirectoryReader reader = DirectoryReader.open(dir)) {
        SortedSetDocValues values = DocValues.getSortedSet(reader.leaves().get(0).reader(), "f");
        values.nextDoc();
        values.intoBitSet(1, new FixedBitSet(1), 0);
        // now on doc 1 ("b", ord 1): nextOrd() should return 1 but returns doc 0's ord 0 ("a")
        assertEquals("nextOrd() on doc " + values.docID(), 1, values.nextOrd());
      }
    }
  }
}

Command

cp $REPRO/TestSingletonSortedSetIntoBitSetStaleOrd.java $LUCENE/lucene/core/src/test/org/apache/lucene/index/
cd $LUCENE && ./gradlew :lucene:core:test --tests org.apache.lucene.index.TestSingletonSortedSetIntoBitSetStaleOrd -Ptests.seed=DEADBEEF -Ptests.asserts=true
# also with -Ptests.asserts=false

Expected: nextOrd() returns 1, the ordinal of doc 1's value "b".
Observed: nextOrd() returns 0, doc 0's "a": nextOrd() on doc 1 expected:<1> but was:<0>, with assertions on and off.

All files of this repro: repro/LFV-091/

More files, environment, full output333 words

The test sits in org.apache.lucene.index like the upstream doc-values tests but uses only public classes and methods (no package-private API; intoBitSet itself is @lucene.internal, and the test uses the test framework's LuceneTestCase and TestUtil). It pins the default codec and a ByteBuffersDirectory; closing the writer flushes one segment. It is deterministic: seeds DEADBEEF (gradle and the standalone runner, assertions on and off) and 5EED1234 (standalone, assertions off) fail the same way. With the recommended fix it passes.

Suggested regression test: this same test, for example as a method of TestDocValues.

Caveat: the test asserts neither that the index has one segment nor that it got the single-valued view. On the default codec at 645d4fe14a20 both hold (closing the writer flushes one segment; an independent verifier's test asserts the view), but a future change could make the test pass without exercising SingletonSortedSetDocValues. For the upstream regression test, add assertEquals(1, reader.leaves().size()); right before and assertNotNull(DocValues.unwrapSingleton(values)); right after getSortedSet: with both lines the test still fails the same way on main, with assertions on and off, and passes with the recommended fix, assertions on and off.

The visible consequence (the extra hit from forOrdinalRange on 16,384 skip-indexed documents) is deliberately not part of the repro; the hunt's range test can be shared if a maintainer asks for it. On 10.3.0-10.4.0 the factory does not exist yet, but the root cause reproduces (this test on 10.3.0, its two-document twin on 10.3.0-10.4.0); 10.2.x passes, and earlier releases lack intoBitSet.

Environment

lucene	`main 645d4fe14a20 (11.0.0-SNAPSHOT); the root cause also on 10.3.0-10.6.0`
jdk	`OpenJDK 25.0.3 (build 25.0.3+9-LTS)`
os	`Linux, aarch64 (the only platform run; the code path is pure Java)`

Observed output

TestSingletonSortedSetIntoBitSetStaleOrd > testNextOrdAfterIntoBitSet FAILED
    java.lang.AssertionError: nextOrd() on doc 1 expected:<1> but was:<0>
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:2E4DFF097067A00B]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.failNotEquals(Assert.java:835)
        at org.junit.Assert.assertEquals(Assert.java:647)
        at org.apache.lucene.index.TestSingletonSortedSetIntoBitSetStaleOrd.testNextOrdAfterIntoBitSet(TestSingletonSortedSetIntoBitSetStaleOrd.java:45)
:lucene:core:test (FAILURE): 1 test, 1  failure

# the same with -Ptests.asserts=false, and on the standalone runner with -ea and -da
# with the recommended fix: 1 test, 0 failures

Why it looks unintended

intoBitSet must behave exactly like a nextDoc() loop, which refreshes the view's ordinal; nextOrd() is for the current document.
The asserting test wrapper resets its ordinal state after intoBitSet, consistent with allowing nextOrd() there, and the codec's own sorted-set iterators refresh their state there.
#16714 fixed the same omission in the SortedSetSelector wrappers; 10.2.2, before this override, is correct.

The full argument436 words

Contract. DocIdSetIterator#intoBitSet "should behave exactly as if implemented as" the docID()/nextDoc() loop (DocIdSetIterator.java:187-194); through that loop the view refreshes ord. SortedSetDocValues#nextOrd: "Returns the next ordinal for the current document" (SortedSetDocValues.java:35-36).
The test framework expects this call. AssertingSortedSetDocValues#intoBitSet delegates, then sets lastDocID = docID() and ordsRetrieved = 0 (AssertingLeafReader.java:1313-1323), so it resets its per-document ordinal state, consistent with reading the new current document's ordinals after intoBitSet (there is no prose contract saying so). By git blame that code came with #14529, by the author of #14618, two weeks earlier.
The codec's own sorted-set iterators refresh their state in intoBitSet: dense multi-valued calls advance(upTo) (Lucene90DocValuesProducer.java:2373-2380), sparse multi-valued sets set = false (:2433-2436).
Same omission, fixed in a sibling. #16714 ("Make SimpleCodec return docvalues singletons", by dweiss, merged 2026-09-25, milestone 10.6.0) added setOrd() after in.intoBitSet(...) in all four SortedSetSelector wrappers (SortedSetSelector.java:135-138, :215-218, :299-302, :384-387). The PR body says "it also did find a bug in there so it's piggybacked"; asked in review what the bug was, its author explained that these classes "keep a pointer to the current doc's ordinal (in a private field) [...] But the intoBitSet method just delegated the call and didn't update this ordinal properly." SingletonSortedSetDocValues was not touched. This is a precedent from a sibling class, not a maintainer statement about SingletonSortedSetDocValues.
The other wrapper in #14618 is safe because it has no cache. SingletonSortedNumericDocValues delegates intoBitSet the same way (SingletonSortedNumericDocValues.java:67-68) but reads in.longValue() on demand (:90). #14618 had one approval and no review comments.
No unwrap precondition. The forOrdinalRange(SortedSetDocValues, ...) javadoc (DocValuesRangeIterator.java:109-116) does not ask callers to unwrap singletons; the sandbox query and the test framework (BaseDocValuesFormatTestCase.java:1193-1226) pass the reader's view. #16172 presents the core unwrap as a speed-up: "Single-valued fields see the largest gain (2.4–3.4x) because the singleton unwrap avoids multi-valued ordinal iteration in YES_IF_PRESENT/MAYBE blocks".
Regression. 10.2.2, without the override, returns the right ordinal; swapping only this file between its pre- and post-#14618 versions flips the result both ways.
Why tests missed it. The ordinal-range intoBitSet tests in TestDocValuesRangeIteratorIntoBitSet use the SortedDocValues overload (:623-626, :728-732); the test-framework check that does pass the reader's SortedSetDocValues view tests docIDRunEnd(), not intoBitSet (BaseDocValuesFormatTestCase.java:1193-1240); and the test framework's testRandomDenseSortedSetIntoBitSet / testRandomSparseSortedSetIntoBitSet (:997-1016, via doTestRandomIntoBitSet, :1041-1095) add two values per document, so they do not get the single-valued view, and they compare only docID() and the bits, never nextOrd(). Not an exhaustive audit of the tests.

The devil's-advocate check153 words

Strongest case (the intended-lens reviewers and the adjudication): intoBitSet is @lucene.internal and its javadoc talks only about which doc IDs get loaded; DocValuesRangeIterator is @lucene.experimental; both core callers unwrap the singleton first, so the SortedSetDocValues overload could be read as meant for truly multi-valued iterators.

Why it fails: "behave exactly as if implemented as" the loop covers the iterator's state after the loop; the asserting test wrapper resets its ordinal state after intoBitSet, consistent with allowing nextOrd() there; the codec's own sorted-set iterators and, since #16714, the SortedSetSelector wrappers refresh their state; nothing documents an unwrap precondition, and a caller cannot know it got the singleton without checking, because the reader returns it by default. The annotations lower severity, not validity. The cross-model review rated the intended-behaviour argument weak; the intended-behaviour check concluded that it does not survive, and the adjudication rejected it on the same contract, implementation and reach evidence.

Suggested fix

Recommended Refresh the cached ordinal in SingletonSortedSetDocValues#intoBitSet

After in.intoBitSet(...), refresh the cached ordinal unless the iterator is exhausted (ord = in.ordValue(), +4 lines in SingletonSortedSetDocValues), as #16714 did for the SortedSetSelector wrappers. With it the test passes, and so do the doc-values, range-iterator and codec suites (262 tests in a re-run, 532 in the first run), assertions on and off.

  @Override
  public void intoBitSet(int upTo, FixedBitSet bitSet, int offset) throws IOException {
    in.intoBitSet(upTo, bitSet, offset);
    // the wrapped iterator moved: refresh the cached ordinal, as nextDoc/advance do
    if (in.docID() != NO_MORE_DOCS) {
      ord = in.ordValue();
    }
  }

All options and trade-offs4 options · 552 words

Two fixes were built and run in scratch worktrees of 645d4fe14a20: refresh the cached ordinal after intoBitSet (recommended, the same shape as #16714) and drop the cache. Two narrower or costlier options are listed for completeness.

Refresh the cached ordinal in SingletonSortedSetDocValues#intoBitSet (validated by the checks) (recommended)

After in.intoBitSet(...), set ord = in.ordValue() unless the iterator is exhausted (SingletonSortedSetDocValues.java, +4 lines), like nextDoc and advance already do and like #16714's setOrd() in the SortedSetSelector wrappers.

  @Override
  public void intoBitSet(int upTo, FixedBitSet bitSet, int offset) throws IOException {
    in.intoBitSet(upTo, bitSet, offset);
    // the wrapped iterator moved: refresh the cached ordinal, as nextDoc/advance do
    if (in.docID() != NO_MORE_DOCS) {
      ord = in.ordValue();
    }
  }

Trade-offs

Smallest change; keeps #14618's bulk path and the cache's design.
Cost: one ordValue() read per intoBitSet call, not per document. No benchmark was run.

What the checks validated

This version (2026-10-30, fresh scratch worktrees, gradle, seed DEADBEEF): stock + test fails with assertions on and off (expected:<1> but was:<0>); + fix: the test, TestDocValues, TestDocValuesRangeIterator, TestDocValuesRangeIteratorIntoBitSet, TestSortedSetSelector, TestDocValuesRewriteMethod and TestLucene90DocValuesFormat: 262 tests, 28 skipped, 0 failures, with assertions on and off.
First version: stock + test fails through gradle with assertions on and off (expected:<1> but was:<0>); + fix, 14 suites (the test, TestDocValues, TestExitableDirectoryReader, TestDocValuesRangeIterator, TestDocValuesRangeIteratorIntoBitSet, TestDocValuesRewriteMethod, TestSkipBlockRangeIterator, TestSkipBlockRangeIteratorBitSetOperations, TestSortedSetSelector, TestSortedSetSortField, TestDocValuesQueries, TestSortedSetDocValuesSetQuery, TestLucene90DocValuesFormat, TestLucene90DocValuesFormatVariableSkipInterval): 532 tests, 43 skipped, 0 failures, with assertions on and off; :lucene:core:check -x test passes (format, ecjLint, forbiddenApis).
The standalone runner passes with the fixed class shadowed, assertions on and off.
In the hunt the same change made the finder's, the verifiers' and the cross-model reviewers' tests pass, as well as the 9 reach-probe tests and 64 upstream tests.
Not run: the full Lucene test suite, the nightly tests, any benchmark.

Drop the cache: nextOrd() returns in.ordValue() (validated by the checks)

Remove the ord field; nextOrd() returns in.ordValue(), and nextDoc, advance and advanceExact just delegate (+5/-18). The view then cannot go stale after any present or future bulk method. The stale open PR #14475 makes the same change as part of a larger refactor.

Trade-offs

The ordinal is decoded when nextOrd() is called instead of on every positioning: bulk loads and counts skip the read, but each nextOrd() call decodes it. nextOrd() now throws IOException, which the abstract method already declares. No benchmark was run.

What the checks validated

Stock + this fix instead: the same 14 suites, 532 tests, 43 skipped, 0 failures, with assertions on (not run with assertions off or through check).

Unwrap singletons inside forOrdinalRange(SortedSetDocValues, ...)

Call DocValues.unwrapSingleton(values) in the factory and use the SortedDocValues path when it returns non-null, as the core queries already do. Not built.

Trade-offs

Fixes the visible extra hits for every caller and is the path #16172 measured as faster, but leaves the root cause: any other caller that reads nextOrd() after intoBitSet still gets a stale ordinal. Worth doing in addition, not instead.

Remove the intoBitSet override

Fall back to the inherited nextDoc() loop, as before #14618. Not built.

Trade-offs

Correct (10.2.2 behaves this way) but gives up #14618's bulk loading for single-valued sorted-set fields, which the range iterator relies on for speed.

Downstream note

An application that builds an access-restriction filter on forOrdinalRange over the raw single-valued view could let through at most one document per qualifying skip-block boundary; no core query or known downstream product builds filters that way.

How it was found

Reading the iterator state while modelling the bulk range protocol (a YES_IF_PRESENT skip block followed by a MAYBE block) suggested the stale ordinal. A bounded-exhaustive check on the real codec confirmed it: 137 wrong answers with the single-valued view, none once it is unwrapped. It was then reduced to a two-document test.

The method in full133 words

Method: semi-formal (a code-reading hypothesis witnessed by bounded-exhaustive translation validation), a hunt unit, 2026-10-25, on main ea57360912ed. The harness DvRangeExhaustive indexes every assignment of {missing, a, b, c} to 5 documents with a skip interval of 2, runs forOrdinalRange for every ordinal range through the bulk paths (intoBitSet, applyMask) and compares with a brute-force reference: 1,024 configurations, 164,196 checks, 137 mismatches when the single-valued view is passed as is, 0 when it is unwrapped. The first witness, values [a, -, a, b, a] with range [a, a], marks doc 3 ("b").

The finder then wrote the JUnit repro and a cross-model refute review ran before the candidate was filed; verifiers and an adjudication followed. On 2026-10-29 the repro was re-run on main 645d4fe14a20 and minimized to the two-document test.

Prior reports, history and checks

Prior reports

No prior report found in GitHub issues, PRs and commits (Lucene and three downstream projects), the Lucene mailing lists, the JIRA archive and the web, searched 2026-10-26 and refreshed twice on 2026-10-30. The closest item is #16714, which fixed the same omission in the SortedSetSelector wrappers.

Searches6 searches

Context, not reports: #16714 (the sibling fix) and #16126 (which introduced that sibling defect); #14618 introduced this one, with no discussion of the cached ordinal; open stale PR #14475 would remove the cache as a side effect of a large refactor and never mentions it; open PR #16869 would, by code reading only, hide the forOrdinalRange consequence but not the root cause; #16450 (fixed in 10.5.1) is the same iterator family with a different root cause; #16172 and #15989 built the batched ordinal-range path. Search limits: keyword search only; GitHub search does not index PR diffs (open PRs were checked by changed files); a search cannot prove that something was never reported.

What was searched, when, and what turned up
Where	When	How	Found
GitHub (apache/lucene issues, PRs, commits; Elasticsearch, OpenSearch, Solr)	2026-10-26	four scripted prior-report searches (48 issue/PR, 9 commit and 6 downstream queries on class, method and symptom terms) and 54 manual read-only queries such as "stale ordinal", "nextOrd after intoBitSet", "singleton intoBitSet", "setOrd intoBitSet"; review threads of 31 related issues and PRs	no report; only the PRs that built these paths and #16714
Open apache/lucene PRs	2026-10-26	changed-file lists of all 412 open PRs	only #14475 touches the class (removes the cache in a refactor); #16869 and #16897 touch the range iterator, not the view
Mailing lists, JIRA archive, web	2026-10-26	85 queries over dev@, java-user@, issues@, commits@ and builds@; 16 JIRA-archive queries; 20 web queries	only notification mirrors of the PRs above; JIRA has only 2013-2016 API changes (the bug postdates JIRA)
Refresh	2026-10-30	seven issue/PR searches (class name, intoBitSet with nextOrd, ordValue or singleton, forOrdinalRange with singleton or extra hit); open PRs by class name; upstream file blobs	no new item; the file is unchanged on main, branch_11x, branch_10_6 and branch_10_5
Refresh 2 (GitHub API)	2026-10-30	issue/PR searches "SingletonSortedSetDocValues", "intoBitSet nextOrd", "SingletonSortedSetDocValues intoBitSet", "forOrdinalRange singleton"; open PRs by class name; the file's blob and commit history on main and branch_10_6	no new item: #15955, #15596, #15654, #14882 (an unrelated Dense#intoBitSet fix), #14618 and older items, none of which reports this stale-ordinal behaviour; the file is unchanged on main 208608f7530 and branch_10_6, last touched by #14618
Refresh 3 (GitHub API, by the second fact check)	2026-10-30	issue/PR searches "SingletonSortedSetDocValues", "intoBitSet ordinal", "stale ord"; the file's blob and last commit on main and branch_10_6; the open PRs #14475, #16869 and #16897	no new report; the file is unchanged on main 2f424fd428d and branch_10_6, last touched by #14618; none of the three open PRs touches the class

History

Introduced in 10.3.0 by #14618 (cebfb0f415b, merged 2025-05-09), which added the forwarding intoBitSet override. The root cause is in every release since 10.3.0, the latest as of 2026-10-30, 10.6.0, included; the forOrdinalRange extra hits since 10.5.0 (#15989, #16172). 10.2.x is not affected.

Versions tested9 rows

#14618 "Impl intoBitset and docIdRunEnd for wrappers" (by gf2121, approved by jpountz with no comments) is cebfb0f415b on main and f2319c52ea9 on branch_10x, merged 2025-05-09, first released in 10.3.0 (CHANGES 10.3.0, Optimizations). Before it the class inherited the default intoBitSet loop, which calls the view's own nextDoc(). Swapping only SingletonSortedSetDocValues.java between its pre- and post-#14618 versions flips the result both ways on 10.2.2, 10.3.0, 10.5.2 and main; no build at the parent commit was made. The visible consequence also needs the forOrdinalRange(SortedSetDocValues, ...) factory (#15989) and the batched sorted-set path that calls disi.intoBitSet on YES_IF_PRESENT blocks (#16172, merged 2026-06-05), both first released in 10.5.0; they rely on the contract #14618 broke and are not themselves the defect.

Versions and branches executed, with results
Version or commit	Verdict	Result
main `645d4fe14a20` (11.0.0-SNAPSHOT)	affected	the test fails, assertions on and off (gradle and standalone); the range test also fails (re-verified 2026-10-29)
upstream main `208608f7530` (2026-10-30), branch_10_6; earlier also branch_11x, branch_10_5	read	same `SingletonSortedSetDocValues.java` blob (read through the GitHub API 2026-10-30, again on main `2f424fd428d`; branch_11x and branch_10_5 heads as fetched 2026-10-29, branch_10_5's head dated 2026-10-04)
10.6.0 (latest release as of 2026-10-30)	affected	the minimal test fails with assertions on and off; the finder's range test also fails (assertions off), extra hit doc 8192 (fact check, 2026-10-30)
10.5.0, 10.5.1, 10.5.2	affected	root cause and range extra hit both fail (10.5.2 also with assertions off)
10.3.0, 10.3.1, 10.3.2, 10.4.0	root cause only	the two-document check fails (on 10.3.0 also this doc's minimal test, assertions on and off); no `forOrdinalRange(SortedSetDocValues, ...)` factory yet
10.2.0, 10.2.1, 10.2.2	not affected	passes: no override, the default loop calls the view's `nextDoc()` (10.2.2 also with this doc's minimal test)
10.1.0, 10.0.0, 9.12.3	API absent	the inherited `DocIdSetIterator#intoBitSet` API did not exist yet (added in 10.2.0 by #14069; compile error)
file swap pre/post #14618	bisect	the old file fixes 10.3.0, 10.5.2 and main; the new file breaks 10.2.2
with the fix (main `645d4fe14a20`)	not affected	the test and the doc-values, range-iterator and codec suites pass

Independent checks

The finder produced the candidate; cross-model reviews (OpenAI GPT-5.6, with code execution), more verifiers and an adjudication rated or confirmed it real; the adjudication re-ran it with a new seed. It still reproduces on main 645d4fe14a20, and this doc's runs validated the fix twice.

Checks and verdicts499 words

Every check re-ran a repro on stock jars; the reviewers and verifiers wrote their own tests from the claim. The early checks ran on main ea57360912ed; the file and the range iterator are unchanged on 645d4fe14a20, where the re-verification and this doc's runs reproduced it.

Each independent check and its verdict
Check	Verdict	What it found
finder	candidate	Bounded-exhaustive check: 137 mismatches with the view, 0 unwrapped; repro fails on main (`-ea`, `-da`), 10.5.2 and branch_10_6; root-cause test fails on 10.3.0, passes on 10.2.2.
cross-model review, refute lens (OpenAI GPT-5.6)	real, high	Own root-cause test fails with `-ea`, `-da` and several seeds; own `forOrdinalRange` test marks a false positive; controls pass; the fix passes its tests and 43 existing ones.
Independent reproducer	real	`nextOrd()` after `intoBitSet` disagrees with the `nextDoc()` loop in 367 of 500 and 372 of 499 random windows; `forOrdinalRange` and `applyMask` mark doc 8192; count 4,151 instead of 4,150; a one-line fix makes all its tests pass.
Cross-model review, intended lens (OpenAI GPT-5.6)	real, high	Works-as-designed case weak; own test fails on main and 10.3.0, passes on 10.2.2 and with the fix.
Devil's advocate	real, high	No intended-behaviour argument survives; own test on the `LeafReader` view fails; a public-class-only filter miscounts in a `FILTER` conjunction.
adjudication	confirmed, high	New seed 1A2B3C4D: 2 of 2 fail with `-ea` and `-da`, 0 of 2 with the fix.
re-verification on main 645d4fe14a20	still reproduces	Finder repro 2 of 2, security probe 4 of 9 and the independent reproduction range tests 2 of 5 fail, same messages.
this doc's runs (two versions)	reproduced, fix validated	The test fails through gradle and the standalone runner, assertions on and off; both fixes pass the suites listed under Suggested fix. Re-run for this version in fresh scratch worktrees: the test fails stock with assertions on and off; with the recommended fix it and 6 related suites pass (262 tests, 0 failures, assertions on and off).
fact checks of the first doc version (2026-10-30)	reproduced; corrections applied	Re-ran the doc's gradle command (assertions on and off) and the standalone runner on main, ran the 10.6.0 release jars (they fail), and pinned the stale ordinal to doc 0 with a probe; an independent reviewer from another model family (OpenAI GPT-5.6) checked the claims against the sources. Their corrections are applied. This version keeps every correction.
fact checks of the second doc version (2026-10-30)	reproduced; small corrections applied	Ran the doc's gradle command in its own scratch worktree (assertions on and off: 1 test, 1 failure, the observed output verbatim), the standalone runner on main (fails; passes with the fix), the minimal test on the 10.6.0 and 10.3.0 (fail) and 10.2.2 (pass) release jars, and a probe through `DocValuesRangeIterator` (the stale ordinal is doc 0's); re-checked upstream. The independent reviewer from another model family checked the claims again, and the diagram check found no errors. Their corrections are applied.

Diagram file

The diagram as a still image, generated with AI and checked against the code. It is under 10 MB, the limit for a GitHub attachment.

PNG 598 KB

Sources and provenance

Finding record, Lucene and upstream sources, repro files and diagram notes1,397 words

Finding record

LFV-091, snapshot of 2026-11-06

Registry title: SingletonSortedSetDocValues.intoBitSet does not refresh its cached ordinal, so nextOrd() returns an earlier document's ordinal and DocValuesRangeIterator.forOrdinalRange can return extra hits on single-valued segments
Area: index / doc values (single-valued SortedSetDocValues view, DocValuesRangeIterator)
Priority: Low, rank 593: Moved down from Medium: SingletonSortedSetDocValues.intoBitSet leaves a stale ordinal so skip-indexed forOrdinalRange filters return a few extra documents, but only custom code on internal or experimental APIs (or a sandbox query) reaches it, because core queries unwrap the singleton.
Impact class: silent wrong results
Impact note: Hits only code that reads ordinals after SortedSetDocValues#intoBitSet (@lucene.internal) on a single-valued segment, or that passes the reader's single-valued SortedSetDocValues view of a skip-indexed field (opt-in SortedSetDocValuesField.indexedField) without unwrapping it to the experimental DocValuesRangeIterator.forOrdinalRange (or a contiguous forOrdinalSet) and evaluates it in bulk (count, sort-by-field or no-scores FILTER conjunctions; top-scores searches were correct), plus the out-of-scope sandbox SortedSetDocValuesMultiRangeQuery when its ranges collapse to one; no core Lucene query is affected because they unwrap the singleton first; default codec, ordinary scale, a few extra hits per segment; the root cause is released since 10.3.0 and the range-iterator consequence since 10.5.0.
Summary: You take the SortedSetDocValues of a segment whose documents have at most one value each (DocValues.getSortedSet and LeafReader#getSortedSetDocValues then return a single-valued wrapper), call intoBitSet(upTo, ...), and read nextOrd() for the document the iterator now sits on; or you pass that view to DocValuesRangeIterator.forOrdinalRange on a skip-indexed field and load the matches in bulk. You expect nextOrd() to describe the current document, as after the nextDoc() loop that the intoBitSet javadoc says it behaves exactly like, and the range iterator to mark only documents whose value is in range. Instead the wrapper's intoBitSet moves the inner iterator without refreshing its cached ordinal, so with two documents a and b nextOrd() on doc 1 returns a's ordinal, and forOrdinalRange for [b, b] also marks doc 8192, whose value is a.
Introduced: cebfb0f415b4 (#14618 'Impl intoBitset and docIdRunEnd for wrappers', main; branch_10x backport f2319c52ea90), which added the delegating intoBitSet override [exec: release bisection 10.2.2 pass / 10.3.0 fail, and swapping only SingletonSortedSetDocValues.java between its pre- and post-#14618 versions flips the result both ways on 10.2.2, 10.3.0, 10.5.2 and main; no build at the parent commit]. Regression: before it the inherited DocIdSetIterator#intoBitSet loop called the wrapper's own nextDoc(), which refreshes the ordinal. The visible forOrdinalRange extra hit also needs the SortedSetDocValues factory (#15989) and the batched sorted-set range path (#16172), both first released in 10.5.0; they rely on the intoBitSet contract that #14618 broke and are not themselves the defect [read] (2025-05-09 (#14618 merged); range-iterator consequence reachable from 2026-06-05 (#16172 merged))
First affected version: 10.3.0 (root cause) [exec]; 10.5.0 for the forOrdinalRange extra hit [exec]
Affected versions: root cause: 10.3.0, 10.3.1, 10.3.2, 10.4.0, 10.5.0, 10.5.1, 10.5.2, branch_10_6 d04ee7317726 (10.6 line; 10.6.0 released 2026-10-29) and main ea57360912ed [exec: root-cause JUnit on every listed release; 10.3.0 and main also with -da]; forOrdinalRange extra hit: 10.5.0, 10.5.1, 10.5.2, branch_10_6 and main [exec, 10.5.2 and main also with -da]; 10.4.0 has no such factory (the full test does not compile) [exec]; not affected: 10.2.0-10.2.2 (no override, the default loop calls the wrapper's nextDoc) [exec pass]; not applicable: 10.1.0 and earlier, including 9.12.3 (SortedSetDocValues#intoBitSet absent, compile error) [exec]; the file is unchanged since #14618 on upstream main, branch_10x and branch_10_6 [read, 2026-10-25]; main 645d4fe14a2 [exec 2026-10-29, -ea and -da]; SingletonSortedSetDocValues.java has the same blob on main 645d4fe14a2, branch_10x 657014b3c57 and branch_10_6 4ae946e5a9b (10.6.0 RC1) [read 2026-10-29]
Status: documented (brief, checks and a per-finding doc exist)
Verification: Confirmed by AI-run checks: an independent repro, a cross-model review that agreed, an intended-behaviour check, a prior-report search and a skeptical audit; not yet triaged by a person.
Found by: semi-formal: exhaustive check, code reading
Upstream and prior reports: none found (2026-10-26: a scripted prior-report search, 4 runs with 48 GitHub issue/PR, 9 commit, 6 downstream-repo, 85 mailing-list (dev@, java-user@, issues@, commits@, builds@) and 16 JIRA-archive queries; 54 manual read-only gh searches, including Elasticsearch, OpenSearch and Solr; the review threads of 31 related issues and PRs; the changed-file lists of all 412 open apache/lucene PRs; 20 web queries); related prior items, none of them this bug: #16714 (sibling: fixed the identical omission in the four SortedSetSelector wrappers, did not touch or mention SingletonSortedSetDocValues; the precedent), #14618 (introduced it, no discussion of the cached ordinal), #16126 (introduced the sibling defect), open stale PR #14475 (a large refactor that would remove the cache as a side effect, never mentions it), open PR #16869 (by code reading would hide the forOrdinalRange consequence, not the root cause), #16450 (same iterator family, different root cause, fixed in 10.5.1), #16172 and #15989 (built the batched ordinal-range path)
Lucene commit tested: main 645d4fe14a20 (11.0.0-SNAPSHOT, commit of 2026-10-28): the repro test through gradle in a scratch worktree (assertions on and off), stock and with each fix, plus 13 doc-values, range-iterator and codec suites with each fix; the standalone runner with -ea (seed DEADBEEF) and -da (seeds DEADBEEF and 5EED1234), and with the recommended fix shadowed. Release jars (SHA-1 checked) 9.12.3, 10.0.0, 10.1.0, 10.2.0-10.2.2, 10.3.0-10.3.2, 10.4.0, 10.5.0-10.5.2 and a branch_10_6 build, with the finder's root-cause test and range test (2026-10-26); the 10.6.0 release jars (SHA-1 checked, 2026-10-30): the minimal test with -ea and -da, the finder's two tests with -da. Upstream heads as fetched 2026-10-29, read 2026-10-30: SingletonSortedSetDocValues.java has the same blob on main 716d6f216f5, branch_11x, branch_10_6 and branch_10_5. Only the default codec, Linux aarch64 and OpenJDK 25.0.3 were run. Again on 2026-10-30 (this version): fresh scratch worktrees of 645d4fe14a20, the test through gradle stock (assertions on and off) and with the recommended fix plus six doc-values, range-iterator and codec suites (assertions on and off); upstream main 208608f7530 and branch_10_6 read through the GitHub API the same day (same SingletonSortedSetDocValues.java blob). The second fact check of the same day ran the doc's gradle command in its own scratch worktree (assertions on and off), the standalone runner on main (stock and with the fix), the minimal test on the 10.6.0, 10.3.0 and 10.2.2 release jars (assertions on and off), and re-read upstream main 2f424fd428d and branch_10_6 (same blob).

Lucene and upstream sources

Finding record: lucene/core/src/java/org/apache/lucene/index/SingletonSortedSetDocValues.java:55-57 (nextOrd returns the cached ord), :65-89 (nextDoc/advance/advanceExact refresh it), :91-94 (intoBitSet only delegates to in.intoBitSet)
lucene/core/src/java/org/apache/lucene/search/DocIdSetIterator.java:185-194 (intoBitSet 'should behave exactly as if implemented as' the nextDoc() loop), :203 (@lucene.internal)
lucene/core/src/java/org/apache/lucene/index/SortedSetDocValues.java:35-36 (nextOrd 'Returns the next ordinal for the current document')
lucene/core/src/java/org/apache/lucene/search/DocValuesRangeIterator.java:37 (@lucene.experimental), :108-131 (forOrdinalRange(SortedSetDocValues, ...), no unwrap precondition), :349-369 (YES_IF_PRESENT block: disi.intoBitSet), :537-547 (MAYBE block evaluates the nextOrd() predicate from disi.docID())
lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java:2284-2285 (getSortedSet returns DocValues.singleton(...) for single-valued segments)
lucene/core/src/java/org/apache/lucene/document/SortedSetDocValuesRangeQuery.java:159-164 and lucene/core/src/java/org/apache/lucene/search/DocValuesRewriteMethod.java:168-174 (core callers pass the unwrapped SortedDocValues when it exists)
lucene/test-framework/src/java/org/apache/lucene/tests/index/AssertingLeafReader.java:1313-1324 (AssertingSortedSetDocValues.intoBitSet resets ordsRetrieved = 0 after delegating)
SECURITY.md:28, 50 (threat model)
The finding in 30 seconds: a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/index/SingletonSortedSetDocValues.java:31, :49-62, :64-89, :91-94
a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/search/DocIdSetIterator.java:186-210
a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/index/SortedSetDocValues.java:35-43
a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/index/DocValues.java:168-183, :325-336
a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java:2282-2286, :2373-2380, :2433-2436
a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesFormat.java:226 (DEFAULT_SKIP_INDEX_INTERVAL_SIZE = 4096)
a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/search/DocValuesRangeIterator.java:109-133, :246-265, :349-371, :404-408, :537-548
a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/search/SkipBlockRangeIterator.java:90-98
a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/search/BooleanScorerSupplier.java:440-446; DenseConjunctionBulkScorer.java:55-59, :278-305; ConstantScoreBulkScorer.java:136-142
a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesConsumer.java:250-262 (when a skip interval closes)
Who is affected: a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/document/SortedSetDocValuesRangeQuery.java:122-123, :159-164
a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/search/DocValuesRewriteMethod.java:165-174
a Lucene main checkout (645d4fe14a20) lucene/sandbox/src/java/org/apache/lucene/sandbox/search/SortedSetDocValuesMultiRangeQuery.java:202-206
a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/search/ConstantScoreWeight.java:27, ConstantScoreScorer.java:27, Weight.java:205 (@lucene.internal)
Why it looks unintended: a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/search/DocIdSetIterator.java:186-210
a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/index/SortedSetDocValues.java:35-36
a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/search/SortedSetSelector.java:135-138, :215-218, :299-302, :384-387
a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/index/SingletonSortedNumericDocValues.java:67-68, :90
a Lucene main checkout (645d4fe14a20) lucene/test-framework/src/java/org/apache/lucene/tests/index/BaseDocValuesFormatTestCase.java:1193-1240
a Lucene main checkout (645d4fe14a20) lucene/core/src/test/org/apache/lucene/search/TestDocValuesRangeIteratorIntoBitSet.java:623-626, :728-732
Suggested fix: a Lucene main checkout (645d4fe14a20) lucene/core/src/java/org/apache/lucene/index/SortedSetDocValues.java:43 (abstract nextOrd() throws IOException)
History: https://github.com/apache/lucene/pull/14618; https://github.com/apache/lucene/pull/16172; https://github.com/apache/lucene/pull/15989
lucene/CHANGES.txt at 645d4fe14a20 (10.2.0: GITHUB#14069 added DocIdSetIterator#intoBitSet)

Other statements trace to the hunt's working files (models, logs, reviews and briefs), which are not published.

Repro files

TestSingletonSortedSetIntoBitSetStaleOrd.java (lucene/core test, one method): repro/LFV-091/src/org/apache/lucene/index/TestSingletonSortedSetIntoBitSetStaleOrd.java (49 lines)

About the diagrams

The PNG is the summary card. Made in parallel with this doc; every element was checked against the code at 645d4fe14a20 and executed runs (see the diagram README).

assets/LFV-091/bug.png (598 KB)

Built 2026-11-06 from the hunt's record of this finding and its doc file. Lucene main under test in this hunt: ea57360912ed. Every listed finding was re-run on Lucene main e1faeba08bde on 2026-11-02.

The finding in 30 seconds

Background: ordinals, the single-valued view and skip blocks

The finding, step by step

Who is affected

The mechanism (main 645d4fe14a20)

The consequence, block by block

When it happens and when not

Minimal repro

Environment

Observed output

Why it looks unintended

Suggested fix

Refresh the cached ordinal in SingletonSortedSetDocValues#intoBitSet (validated by the checks) (recommended)

Drop the cache: nextOrd() returns in.ordValue() (validated by the checks)

Unwrap singletons inside forOrdinalRange(SortedSetDocValues, ...)

Remove the intoBitSet override

Downstream note

How it was found

Prior reports, history and checks

Prior reports

History

Independent checks

Diagram file

Sources and provenance

Finding record

Lucene and upstream sources

Repro files

About the diagrams

The mechanism (main `645d4fe14a20`)