With the vector module enabled, NumericDocValues.rangeIntoBitSet marks documents for a reversed, empty doc range

The finding in 30 seconds

The SIMD helper works out where its vector loop ends from the range length, and starts its plain tail loop at that point. A negative length puts that point below toDoc.

Background: the bulk range scan and its SIMD loop

NumericDocValues.rangeIntoBitSet(fromDoc, toDoc, min, max, bits, offset) (new in 10.5.0) marks every doc in [fromDoc, toDoc) whose value is in [min, max]. It is the fast path behind doc-values range filters, and applications can call it directly.
On a dense field of the default codec it goes to a helper that Lucene picks at startup: a plain Java loop by default or, when the JVM has --add-modules jdk.incubator.vector, its Panama (SIMD) helper. That one runs a vector loop over groups of 2, 4 or 8 docs (the lane count) and a plain tail loop for the rest.

Diagram of the bug: 16 documents 0 to 15 with values 0 to 15. The call rangeIntoBitSet(12, 5) asks for the reversed range [12, 5), which holds no document. The SIMD helper computes the length 5 - 12 = -7, rounds it down to a multiple of the lane count, -8, and sets its loop bound to 12 - 8 = 4. The vector loop from 12 to 4 does not run; the tail loop runs from doc 4 up to 5, finds value 4 in range and marks doc 4. The scalar provider marks nothing. — The repro's call `rangeIntoBitSet(12, 5)`: the length -7 rounds down to -8, so the vector loop is skipped and the tail loop starts at doc 4 and marks it, though `[12, 5)` holds no document. Select the still to open it at full size. Generated with AI, checked against the code.

The finding, step by step

Index 16 documents with a NumericDocValuesField f = 0..15 on the default codec, so the field is dense, and run the JVM with --add-modules jdk.incubator.vector, so Lucene uses its SIMD (Panama) code.
Call rangeIntoBitSet(12, 5, Long.MIN_VALUE, Long.MAX_VALUE, bits, 0). The doc range [12, 5) is empty, so you expect no bit set, which is what the plain Java code gives.
The SIMD code takes the length 5 - 12 = -7 and rounds it down to a multiple of its lane count (2, 4 or 8): -8. Its vector loop is skipped, and its plain tail loop starts at 12 - 8 = 4 and runs up to 5.
So doc 4 is marked: bits.cardinality() is 1, not 0, with assertions on or off and no error. Other reversed ranges mark up to lanes - 1 docs just below toDoc; [10, 5) with 4 lanes marks docs 2, 3 and 4.

Who is affected

Only code that itself calls a public bulk range method with a start past its end, on a dense default-codec field, in a JVM with --add-modules jdk.incubator.vector: NumericDocValues.rangeIntoBitSet (new in 10.5.0) or, since 10.6.0, its sorted variants. Lucene's own queries never pass a reversed range. The result is silently wrong by at most lanes - 1 docs.

Mechanism, terms and numbers559 words

NumericDocValues.rangeIntoBitSet(fromDoc, toDoc, minValue, maxValue, bitSet, offset) (public, new in 10.5.0) marks the docs in [fromDoc, toDoc) whose value is in [minValue, maxValue]. For fromDoc > toDoc that range is empty, and the default implementation, the scalar provider and Lucene90's multi-valued sorted-numeric overrides mark nothing. On a dense default-codec field the call goes to the vectorization provider, and with the Panama provider (a JVM started with --add-modules jdk.incubator.vector) PanamaDocValuesRangeSupport.rangeIntoBitSet has no empty-range guard: its loop bound fromDoc + LONG_SPECIES.loopBound(toDoc - fromDoc) rounds the negative length down to a multiple of the lane count and lands below toDoc, so the scalar tail tests and marks up to lanes - 1 docs just below toDoc that were never requested. With 16 docs (values 0..15), rangeIntoBitSet(12, 5, MIN, MAX, bits, 0) marks doc 4, with assertions on or off and no error. No Lucene caller passes a reversed range at 5adff78bb168, so only application code that calls a bulk range method itself can hit it: this one or, since 10.6.0, SortedNumericDocValues.rangeIntoBitSet on a single-valued field and SortedDocValues.ordinalRangeIntoBitSet on a dense sorted field, which reach the same helper.

The mechanism (main `5adff78bb168`)

Entry. The javadoc: "Fills a FixedBitSet with the doc IDs in [fromDoc, toDoc) whose values are in [minValue, maxValue]", fromDoc inclusive, toDoc exclusive, no ordering precondition. The default loop for (int d = fromDoc; d < toDoc; d++) does nothing for a reversed range (NumericDocValues.java:113-140).
Forwarding. The default codec's dense numeric field forwards both bounds unchanged (Lucene90DocValuesProducer.java:1019-1030) to the provider's DocValuesRangeSupport (:533-543). By reading, the gcd/delta dense path (:1066-1076) reaches the same method (not executed). Since 10.6.0 two more public methods reach it on dense fields: SortedNumericDocValues.rangeIntoBitSet on a single-valued field (DocValues.singleton at :2003, then SingletonSortedNumericDocValues.java:80, #16295) and SortedDocValues.ordinalRangeIntoBitSet on a dense sorted field (:1549-1555, #16180). With (12, 5) each marks doc 4 on main, with assertions on and off, and passes with the fix (executed). 10.5.x has neither override.
The fix site. PanamaDocValuesRangeSupport.rangeIntoBitSet (:36-71) has no empty-range check. Line 48: loopBound = fromDoc + LONG_SPECIES.loopBound(toDoc - fromDoc). VectorSpecies.loopBound returns the largest multiple of the lane count that is <= its argument, so a negative length rounds down: loopBound(-7) = -8 with 2, 4 or 8 lanes.
What runs. The SIMD loop (:51, d = fromDoc; d < loopBound) is skipped. The scalar tail (:65-70) starts at loopBound, not at fromDoc: for (int d = loopBound; d < toDoc; d++) reads each value and sets d - offset when it is in range. For [12, 5): loopBound = 12 - 8 = 4, so doc 4 is tested and marked.
In general the tail marks the matching docs in [toDoc - floorMod(toDoc - fromDoc, lanes), toDoc), at most lanes - 1. [10, 5) with 4 lanes marks docs 2, 3 and 4 (executed).
Siblings are no-ops. The scalar provider loops d = fromDoc; d < toDoc (DefaultDocValuesRangeSupport.java:39); Lucene90's multi-valued sorted-numeric override starts with int endDoc = Math.min(toDoc, maxDoc); if (fromDoc >= endDoc) return; (Lucene90DocValuesProducer.java:2083-2086, from #16166). By reading, the Panama sibling sortedNumericRangeIntoBitSet is also a no-op here (integer division rounds toward zero and its tail starts at fromDoc), and its only caller guards first.

Exposure and likelihood263 words

Who. Application code that itself calls a public, untagged bulk range method with fromDoc > toDoc, for example a computed start that has run past its end, on a dense field of the default codec, with the Panama provider: NumericDocValues.rangeIntoBitSet (new in 10.5.0) or, since 10.6.0, SortedNumericDocValues.rangeIntoBitSet on a single-valued field and SortedDocValues.ordinalRangeIntoBitSet on a dense sorted field (all three executed on main).
Not hit. Lucene's own queries: the three bulk range iterators in DocValuesRangeIterator (numeric :452-453, sorted-numeric :483-484, sorted :517-518) all pass from = Math.max(blockStart, docID()) up to blockEnd, which stays ordered on dense fields. The other in-tree calls only forward their own caller's bounds (SingletonSortedNumericDocValues.java:80, inside Lucene90DocValuesProducer). The default scalar provider is unaffected (executed).
Default vs opt-in. The Panama provider is opt-in (--add-modules jdk.incubator.vector); Lucene logs a hint to enable it (VectorizationProvider.java:150-153). The codec and IndexWriterConfig are defaults.
Effect. Extra docs in the caller's bitset, at most lanes - 1, and the count depends on the CPU's vector width. No exception with -ea or -da. No scale needed: 16 docs suffice.
Possible future reach (not executed). Open PR #16897 would add an in-tree caller that passes from = Math.max(approximation.docID(), offset) without checking from < upTo (by reading its diff).
Why tests miss it. No test passes a reversed range. Also, with Lucene's local default test options the scalar provider ran ("Vector bitsize enforcement; using default vectorization provider outside of testMode") and the repro passed; with -Ptests.defaultvectorization=true the Panama provider ran and it failed (executed, seed DEADBEEF).

Minimal repro

One JUnit test on the public API: 16 dense docs, one rangeIntoBitSet(12, 5, ...) call, assert no bit is set. Deterministic, single thread, about 1 s. Run it with the Panama provider (-Ptests.defaultvectorization=true); it fails with assertions on and off.

TestReversedRangeIntoBitSet.java (lucene/core, package org.apache.lucene.index, 1 test)48 lines

/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.apache.lucene.index;

import org.apache.lucene.document.Document;
import org.apache.lucene.document.NumericDocValuesField;
import org.apache.lucene.store.ByteBuffersDirectory;
import org.apache.lucene.store.Directory;
import org.apache.lucene.tests.util.LuceneTestCase;
import org.apache.lucene.tests.util.TestUtil;
import org.apache.lucene.util.FixedBitSet;

public class TestReversedRangeIntoBitSet extends LuceneTestCase {

  public void testReversedRangeSetsNoBits() throws Exception {
    try (Directory dir = new ByteBuffersDirectory()) {
      try (IndexWriter w =
          new IndexWriter(dir, new IndexWriterConfig().setCodec(TestUtil.getDefaultCodec()))) {
        for (int i = 0; i < 16; i++) {
          Document doc = new Document();
          doc.add(new NumericDocValuesField("f", i));
          w.addDocument(doc);
        }
      }
      try (DirectoryReader reader = DirectoryReader.open(dir)) {
        NumericDocValues dv = getOnlyLeafReader(reader).getNumericDocValues("f");
        FixedBitSet bits = new FixedBitSet(16);
        dv.rangeIntoBitSet(12, 5, Long.MIN_VALUE, Long.MAX_VALUE, bits, 0);
        // Expected: [12, 5) is empty, so no bit is set; actual with the Panama provider: doc 4 is set.
        assertEquals(0, bits.cardinality());
      }
    }
  }
}

Command

cp $REPRO/TestReversedRangeIntoBitSet.java $LUCENE/lucene/core/src/test/org/apache/lucene/index/
cd $LUCENE && ./gradlew :lucene:core:test --tests org.apache.lucene.index.TestReversedRangeIntoBitSet -Ptests.seed=DEADBEEF -Ptests.defaultvectorization=true
# and with -Ptests.asserts=false: the same expected:<0> but was:<1>

Expected: bits.cardinality() is 0: [12, 5) contains no doc, as the scalar provider gives.
Observed: expected:<0> but was:<1>: doc 4 is marked, with assertions on and off, and no error.

All files of this repro: repro/LFV-1145/

More files, environment, full output304 words

TestReversedRangeIntoBitSet (one method, extends LuceneTestCase, package org.apache.lucene.index of the lucene/core tests). The length -7 marks exactly one doc with 2, 4 or 8 lanes, so the failure is the same on any CPU that runs the Panama provider; 4 lanes (gradle and the standalone runner) and, with an equivalent test, 2 lanes were executed; x86 was not run.

Without -Ptests.defaultvectorization=true, Lucene's local default test options ran the scalar provider for seed DEADBEEF and the test passed. The comment line is 103 columns, so checkGoogleJavaFormat flags it and proposes a re-wrap onto two lines (formatting only; it suggests ./gradlew tidy, which was not run). As the regression test, the same method fits next to the dense-field cases in TestDocValuesRangeIteratorIntoBitSet or in TestLucene90DocValuesFormat; it only catches the bug in a run that uses the Panama provider.

About TestReversedRangeIntoBitSet.java (lucene/core, package org.apache.lucene.index, 1 test)

The doc range [12, 5) is reversed and so empty; the test asserts that no bit is set.

Environment

lucene	`main 5adff78bb168 (12.0.0-SNAPSHOT): fails; 10.5.0, 10.5.2, 10.6.0 (equivalent tests): fail; 10.4.0, 9.12.3: no rangeIntoBitSet (does not compile)`
jdk	`OpenJDK 25.0.3 (build 25.0.3+9-LTS), with the vector incubator module (Lucene logs "Java vector incubator API enabled; uses preferredBitSize=256")`
os	`Linux aarch64, 256-bit vectors (4 long lanes)`

Command (environment)

cp $REPRO/TestReversedRangeIntoBitSet.java $LUCENE/lucene/core/src/test/org/apache/lucene/index/
cd $LUCENE && ./gradlew :lucene:core:test --tests org.apache.lucene.index.TestReversedRangeIntoBitSet -Ptests.seed=DEADBEEF -Ptests.defaultvectorization=true

Observed output

# gradle, scratch worktree at 5adff78bb168, -Ptests.seed=DEADBEEF -Ptests.defaultvectorization=true -Ptests.asserts=false
TestReversedRangeIntoBitSet > testReversedRangeSetsNoBits FAILED
    java.lang.AssertionError: expected:<0> but was:<1>
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:CEBF79BA90305CCE]:0)
        ...
        at org.apache.lucene.index.TestReversedRangeIntoBitSet.testReversedRangeSetsNoBits(TestReversedRangeIntoBitSet.java:44)
  2> INFO: Java vector incubator API enabled; uses preferredBitSize=256; FMA enabled
:lucene:core:test (FAILURE): 1 test, 1  failure

# same with assertions on: identical failure

# same, Lucene's local default test options (no -Ptests.defaultvectorization):
#   "Vector bitsize enforcement; using default vectorization provider outside of testMode" -> pass

# with the fix: PASS with assertions on and off

Why it looks unintended

The javadoc promises the docs in [fromDoc, toDoc), with no fromDoc <= toDoc precondition; [12, 5) holds none.
The default loop, the scalar provider and Lucene90's multi-valued sorted-numeric overrides (if (fromDoc >= endDoc) return;) all mark nothing.
It depends on a JVM flag: [10, 5) marks no doc with the scalar provider, docs 2, 3, 4 with 4 SIMD lanes.

The full argument124 words

Contract. NumericDocValues.java:113-128 and the internal DocValuesRangeSupport.java:33-44 both define the result as the docs in [fromDoc, toDoc), with no ordering precondition.
Implementations. The default loop (NumericDocValues.java:132), the scalar provider (DefaultDocValuesRangeSupport.java:39) and Lucene90's multi-valued sorted-numeric overrides (Lucene90DocValuesProducer.java:2083-2086, #16166) treat the range as empty; open PR #16835 adds the same fromDoc >= endDoc guard to its new paths.
Nearest conventions. FixedBitSet.set(start, end) returns when end <= start (FixedBitSet.java:777-779); DocIdSetIterator.intoBitSet's documented loop runs only while doc < upTo (DocIdSetIterator.java:187-210).
History. #16050 introduced the API, the forwarding and the unguarded loop together, and its discussion never mentions reversed or empty ranges. Open PR #16917, by the same author, guards fromIndex >= toIndex in its new fast path.

The devil's-advocate check144 words

Strongest case: a reversed half-open range is caller error. Java's List.subList and Objects.checkFromToIndex throw, Lucene's FixedBitSet.cardinality(from, to) checks its range (FixedBitSet.java:227-228) and DocIdSetIterator.range rejects minDoc >= maxDoc (DocIdSetIterator.java:47-51); the method is an optimisation hook, FixedBitSet and DocValuesRangeSupport are @lucene.internal, and no Lucene caller passes such a range. A cross-model reviewer (refute lens) took this view (not real, medium confidence) while reproducing the behaviour and agreeing on the root cause.

Why it fails: no document states the precondition; the method itself is public and untagged; every other implementation treats the range as empty; and the conventions cited above answer an illegal range by throwing or by doing nothing, none by returning docs outside it. The missing caller is why the priority is Low, not why the behaviour is intended. A maintainer may still prefer to reject the input (option B).

Suggested fix

Recommended Return early on an empty range in PanamaDocValuesRangeSupport.rangeIntoBitSet (fix A)

Return early when fromDoc >= toDoc at the top of PanamaDocValuesRangeSupport.rangeIntoBitSet, so it matches the scalar provider: +3 lines. The repro passes with assertions on and off, and 628 tests of the closest doc-values and range suites pass with the Panama provider.

    if (fromDoc >= toDoc) {
      return; // empty range, as in DefaultDocValuesRangeSupport
    }
    final int vectorLen = LONG_SPECIES.length();

All options and trade-offs2 options · 348 words

Option A was built and checked through gradle on 5adff78bb168; B was not built. Open PR #16917 adds a fast path at the top of the same method: if it is merged first, the guard goes before that fast path.

Return early on an empty range in PanamaDocValuesRangeSupport.rangeIntoBitSet (fix A) (validated by the checks) (recommended)

The same no-op that the scalar provider and Lucene90's multi-valued sorted-numeric overrides already give for an empty range.

    if (fromDoc >= toDoc) {
      return; // empty range, as in DefaultDocValuesRangeSupport
    }
    final int vectorLen = LONG_SPECIES.length();

Trade-offs

One comparison per call. fromDoc == toDoc was already a no-op (loopBound(0) = 0). If #16917 is merged first, put the guard before its new fast path (that path guards itself; the old loop it falls back to does not).

What the checks validated

Gradle in a scratch worktree at 5adff78bb168, seed DEADBEEF, -Ptests.defaultvectorization=true: the repro fails without the fix (assertions on and off) and passes with it (on and off).
With the fix: TestLucene90DocValuesFormat (177), TestLucene90DocValuesFormatMergeInstance (177), TestLucene90DocValuesFormatVariableSkipInterval (152), TestSkipBlockRangeIteratorBitSetOperations (56), TestDocValuesRangeIteratorIntoBitSet (40), TestSortedNumericDocValuesRangeIterator (24), TestVectorizationProvider (1) and the repro: 628 tests, 0 failures, 65 skipped by assumptions; each class logged "Java vector incubator API enabled". checkGoogleJavaFormat flags only the repro's comment line. The full core suite was not run.
Earlier runs: the same guard passes the finder's, the independent reproducer's and the cross-model reviewers' tests, and 13 upstream TestDocValuesRangeIteratorIntoBitSet tests (on the older main ea57360912ed).
The same guard covers the single-valued sorted-numeric and dense sorted paths, which reach the same method: a probe of both fails on main without it and passes with it (assertions off).

Reject a reversed range (not built)

Document fromDoc <= toDoc on NumericDocValues.rangeIntoBitSet and assert or throw at the entry.

Trade-offs

Surfaces caller bugs, but changes the default behaviour (today a no-op) and must be applied to every implementation (default loop, scalar and Panama providers, Lucene90 overrides, SORTED and sorted-numeric variants) to give one answer. An assert-only check would still leave -da results provider-dependent, so it complements A rather than replacing it.

Downstream note

Downstream, an application that computes the doc window itself and uses the bitset as a filter would let up to lanes - 1 documents outside the window through; Lucene's own queries never pass such a window.

How it was found

A proof-obligation audit of PanamaDocValuesRangeSupport's contract: the bulk scan must mark exactly the docs of [fromDoc, toDoc) for every int pair the public method accepts. A reversed pair breaks the loop-bound arithmetic; a public-API JUnit test then witnessed the stray bits.

The method in full133 words

Method: semi-formal (a proof-obligation audit of a class's contract, witnessed by an executable repro).

The finder checked PanamaDocValuesRangeSupport.rangeIntoBitSet against the obligation that it selects exactly the doc IDs of the half-open range for every (fromDoc, toDoc) the public NumericDocValues.rangeIntoBitSet accepts. The javadoc states no ordering precondition, and for fromDoc > toDoc the loop bound lands below toDoc.
The finder then wrote a public-API JUnit test ([10, 5), docs 2, 3 and 4 marked) that fails on main with assertions on and off and on 10.6.0 and 10.5.0, a scalar-provider control that passes, and the guard that makes it pass.
An independent reproducer and two executed cross-model reviews reproduced it with their own tests; a devil's-advocate verifier re-ran the finder's test and an adjudication the reproducer's ("Prior reports, history and checks").

Prior reports, history and checks

Prior reports

No prior report in apache/lucene issues, PRs and commits (including the JIRA archive), the mailing lists, downstream trackers or the web (searched 2026-11-02, re-checked 2026-11-03). Open PR #16917 edits the same method but guards only a new fast path.

Searches3 searches

No matching public report was found in the documented searches. Nearest items, none a report: #16050 / #16093 (introduced it), #16123 and #16283 (later edits), open #16917 (guards only its new mmap fast path for 8/16/32/64-bit packed values; the repro's 4-bit values would still take the old loop, by reading the diff), #16166 and open #16835 (empty-range guards in other overrides), open #16897 (a possible future caller), #16896 (an empty value range after the gcd transform, a different root cause). Limits: keyword search; private reports are not visible.

What was searched, when, and what turned up
Where	When	How	Found
GitHub issues, PRs and commits (apache/lucene incl. the migrated JIRA archive)	2026-11-02, re-checked 2026-11-03	two scripted prior-report searches (53 and 51 queries) and 27 manual searches; every issue and PR updated since 2026-10-30; on 2026-11-03, issues and PRs updated since 2026-11-02 matching rangeIntoBitSet or PanamaDocValuesRangeSupport, and every item created since then	no report; only #16835 updated in the area (unrelated guard in other paths)
Open PRs	2026-11-02	the changed files of all 409 open apache/lucene PRs; the diffs of #16917, #16835 and #16897 read	only #16917 touches the fix-site file; it keeps the old loop unguarded
Mailing lists, JIRA, downstream, web	2026-11-02	dev@, java-user@, issues@ and commits@ lists, JIRA-archive queries, Elasticsearch, OpenSearch and Solr trackers, 8 web queries	nothing about this bug

History

Introduced with the API by #16050 (2026-05) and first released in 10.5.0; 10.5.0, 10.5.2 and 10.6.0 fail with the Panama provider, and 10.4.0 has no such method. Not fixed upstream.

Versions tested5 rows

Introduced by #16050 "Add SIMD-accelerated bulk range evaluation for dense numeric doc values" (Sagar Upadhyaya), commit 6f0821d8dda on main (author date 2026-05-20), and its branch_10x copy 84c424ac132 (#16093, 2026-05-21), which shipped in 10.5.0. It added NumericDocValues.rangeIntoBitSet, the Lucene90 dense forwarding and the unguarded loop at once, so no release had the API without the bug (blame of line 48 and :65-70; git log -S on line 48 finds only these two commits; the parents have no rangeIntoBitSet).

Later edits kept the faulty lines: #16123 (1b41317f33d, 2026-05-27) changed only the SIMD-loop body; #16283 (42e71d8e73f, 2026-07-02) added the sibling sorted-numeric method and is the last upstream commit on the file (GitHub commit list, read 2026-11-03). branch_10_6 has the same line 48. #16295 (2026-07-03) and #16180 (2026-07-29), both first released in 10.6.0, routed the single-valued sorted-numeric and the dense sorted bulk methods to the same helper.

Versions and branches executed, with results
Version or commit	Verdict	Result
main `5adff78bb168`	affected	gradle with the Panama provider: fails with assertions on and off; passes with fix A
main `5adff78bb168`, scalar provider	no vector module	passes: no bit set
10.5.0, 10.5.2, 10.6.0	affected	equivalent tests, assertions on (10.6.0 also off): `[10, 5)` marks docs 2, 3, 4 (4 lanes); `[20, 13)` marks 1 doc
10.5.1	by reading	same line 48 and tail loop (not run)
10.4.0, 9.12.3	API absent	no `NumericDocValues.rangeIntoBitSet`: the repro does not compile

Independent checks

A finder, an independent reproducer, two executed cross-model reviews (OpenAI GPT-5.6; the refute lens reproduced it but called the input out of contract), a devil's-advocate verifier, an adjudication, an audit, a final repro check and an independent fact check all reproduced it. This doc's gradle runs reproduced it on main and validated the fix.

Checks and verdicts302 words

The cross-model reviews ran on the older main ea57360912ed, where the fix-site method is the same code. Every other check ran on 5adff78bb168 or the release jars.

Each independent check and its verdict
Check	Verdict	What it found
finder	reproduced	`[10, 5)` test fails on main (assertions on and off), 10.6.0 and 10.5.0 (docs 2, 3, 4); the scalar control passes; the guard passes it.
independent reproducer	real	Own `[20, 13)` test (32 docs) fails on main (on and off, 256- and 128-bit vectors), 10.6.0, 10.5.2 and 10.5.0; the scalar control passes; 10.4.0 and 9.12.3 have no API.
cross-model review, refute lens (OpenAI GPT-5.6, executed)	not real (medium)	Reproduced it (`[9, 4)` marks 3 docs) and agrees on the root cause; objects on contract grounds only (see the devil's-advocate case).
cross-model review, intended lens (OpenAI GPT-5.6, executed)	real (high)	Own `[13, 7)` test fails on main and 10.6.0, passes with the scalar provider and with the guard; 13 upstream tests pass with the guard.
devil's-advocate verifier	real (moderate)	Re-ran the finder's test on main (fail); the strongest "intended or misuse" case is rejected.
adjudication, separate re-runs, audit and final repro check	confirmed, Low	Re-runs fail on main and on the 10.6.0 jars and pass with the guard; the one-test repro fails on main with `-ea` and `-da`.
independent fact check of this doc (executed)	real; 4 text fixes	Re-ran the gradle command and the one-test repro on main (fail with assertions on and off, pass with the fix); found two more public paths to the same helper and the `numBits` behaviour of a too-small bitset; the doc was corrected.
this doc	reproduced; fix validated	Gradle without the fix fails (on and off) with the Panama provider; with it 628 tests pass.

Diagram file

The diagram as a still image, generated with AI and checked against the code. It is under 10 MB, the limit for a GitHub attachment.

PNG 517 KB

Sources and provenance

Finding record, Lucene and upstream sources, repro files and diagram notes1,624 words

Finding record

LFV-1145, snapshot of 2026-11-06

Registry title: With the Panama vector provider, NumericDocValues.rangeIntoBitSet on a dense default-codec field sets bits for a reversed, and therefore empty, doc range [fromDoc, toDoc) with fromDoc > toDoc, where the scalar provider and every other implementation set none (since 10.5.0)
Area: codecs / doc values bulk range scan (Lucene90DocValuesProducer dense paths -> internal vectorization PanamaDocValuesRangeSupport)
Priority: Low, rank 604: With the opt-in Panama vector provider, the bulk NumericDocValues.rangeIntoBitSet silently sets a few stray bits for a reversed, and so empty, doc range in every release since 10.5.0. No Lucene caller passes such a range, so only contrived application code that also enables that provider hits it.
Impact class: minor
Impact note: Hits only application code that calls the public, untagged bulk method NumericDocValues.rangeIntoBitSet (new in 10.5.0) itself with fromDoc > toDoc on a dense field of the default codec, in a JVM started with --add-modules jdk.incubator.vector (the Panama provider: Lucene logs a hint to enable it and its own test JVM enables it); the call then silently sets floorMod(toDoc - fromDoc, lanes) docs just below toDoc whose values are in [minValue, maxValue] (at most lanes - 1; exactly 1 for [20, 13) with 2, 4 or 8 lanes), with -ea and -da, where the scalar provider sets none; no scale is needed, no Lucene caller passes a reversed range at 5adff78bb168 (DocValuesRangeIterator keeps from below blockEnd on dense fields), and since 10.6.0 SortedNumericDocValues.rangeIntoBitSet on a single-valued field and the dense SORTED ordinalRangeIntoBitSet reach the same code and set the same stray doc on main (executed with -ea and -da; the fix removes it), while the gcd/delta dense path does so by reading only (not executed) [exec + read].
Summary: You index 32 documents with NumericDocValuesField values 0..31 on the default codec (a dense field), run the JVM with the vector incubator module enabled, and call NumericDocValues.rangeIntoBitSet(20, 13, Long.MIN_VALUE, Long.MAX_VALUE, bits, 0); you expect no bit to be set, because the javadoc defines the result as the doc IDs in [fromDoc, toDoc), which is empty here, and the scalar provider, the default per-doc loop and Lucene90's own sorted-numeric overrides all set nothing. With the Panama provider doc 12 is set, with no error under -ea or -da. PanamaDocValuesRangeSupport.rangeIntoBitSet has no empty-range guard: loopBound = fromDoc + LONG_SPECIES.loopBound(toDoc - fromDoc) rounds the negative length down to a multiple of the lane count and lands below toDoc, the SIMD loop is skipped, and the scalar tail tests and sets the docs in [loopBound, toDoc) that were never requested (for [10, 5) with 4 lanes: docs 2, 3 and 4).
Introduced: 6f0821d8ddad468057eb1c540ef306d626d8a4d5 on main and its branch_10x copy 84c424ac132ec1d3807f6a0d99fc527be4ac3353 (the one that shipped, in 10.5.0), #16050 / #16093 'Add SIMD-accelerated bulk range evaluation for dense numeric doc values' (sgup432): it added NumericDocValues.rangeIntoBitSet, the Lucene90 dense forwarding and PanamaDocValuesRangeSupport with the unguarded 'loopBound = fromDoc + LONG_SPECIES.loopBound(toDoc - fromDoc)' and the scalar tail 'for (int d = loopBound; d < toDoc; d++)', so the API and the defect arrived together [read: git log -S on that line in the history clone finds only these two commits; blame of PanamaDocValuesRangeSupport.java:48 and :65-70 and Lucene90DocValuesProducer.java:533-543 is 6f0821d8dda; the parents 5ea0f055402 and bd09d6fbad8 have no rangeIntoBitSet; exec: 10.4.0 does not compile the repro, 10.5.0 fails]. Later edits kept the faulty lines: #16123 (1b41317f33d, 2026-05-27) changed only the SIMD-loop body; #16283 (42e71d8e73f, 2026-07-02) added the sibling sortedNumericRangeIntoBitSet, whose only caller guards 'fromDoc >= endDoc' first. (2026-05-20 (author date of 6f0821d8dda, 2026-05-20T03:02:41-07:00; PR #16050 merged 2026-05-20T10:02:41Z); the branch_10x copy 84c424ac132 has author date 2026-05-21 (#16093 merged 2026-05-21T07:43:36Z))
First affected version: 10.5.0 [exec: fails with the Panama provider]; it is the first release with the API, so no release had rangeIntoBitSet without the bug; 10.4.0 and 9.12.3 have no NumericDocValues.rangeIntoBitSet (compile error) [exec]
Affected versions: 10.5.0 through 10.6.0 (released 2026-10-29, 4ae946e5a9b) and main 5adff78bb168 (12.0.0-SNAPSHOT), whenever the JVM runs the Panama provider (--add-modules jdk.incubator.vector). Executed (fail): main with -ea and -da (the finder, the independent reproduction, the intended-behaviour check, the adjudication and a separate re-run), main at 128-bit vectors (the independent reproduction, -XX:UseSVE=0, 2 lanes), 10.6.0 (the finder, the independent reproduction and the history step with -ea; a separate re-run with -da), 10.5.2 (the independent reproduction, history), 10.5.0 (finder, the independent reproduction, history) [exec]. By reading only: 10.5.1 (same line 48 and tail loop), branch_10_6 and branch_11x (same line 48, through the GitHub contents API). Not affected: any version with the default scalar provider [exec: the controls on main and 10.5.0 pass]; 10.4.0 and earlier (no API) [exec 10.4.0, 9.12.3]. Not new in 10.6 (10.5.0 and 10.5.2 fail). Not fixed upstream: the last upstream commit on PanamaDocValuesRangeSupport.java on main is 42e71d8e73f (#16283, author date 2026-07-02); upstream main HEAD and branch_10_6 still have the unguarded line 48; open PR #16917 edits the method but guards only a new fast path that the repro does not take (by reading the diff). Executed on Linux aarch64 with JDK 25.0.3 only (256-bit and 128-bit vector widths); x86 not run.
Status: documented (brief, checks and a per-finding doc exist)
Verification: Confirmed by AI-run checks: an independent repro, a cross-model review that disputed it (the adjudication ruled it real), an intended-behaviour check, a prior-report search and a skeptical audit; not yet triaged by a person.
Found by: semi-formal: proof-obligation audit
Upstream and prior reports: none found (2026-11-02, on top of the finder's, the independent reproduction's, the intended-behaviour check's and the adjudication's searches; rechecked the same day): two scripted prior-report searches (53 and 51 queries over GitHub issues, PRs and commits, downstream repositories, the dev@, java-user@, issues@ and commits@ lists, the JIRA archive and the web; no high-overlap hit besides the introducing PRs #16050 and #16283), 27 manual GitHub searches (apache/lucene, Elasticsearch, OpenSearch, Solr; open, closed and updated-since), the changed-file lists of all 409 open apache/lucene PRs, the discussions of #16050, #16093, #16123, #16166, #16180, #16253, #16283, #16295, #16835, #16869, #16896, #16897 and #16917, and the upstream commit lists of PanamaDocValuesRangeSupport.java on main and branch_10_6: no issue, PR, review comment, list thread or JIRA entry reports a reversed or empty doc range in PanamaDocValuesRangeSupport.rangeIntoBitSet. Nearest, not reports: open PR #16917 (sgup432, created 2026-10-27) edits the fix-site method and guards 'fromIndex >= toIndex' only in its new mmap fast path for 8/16/32/64-bit packed values, so the repro (4 or 5 bits per value) would still fail if it were merged as it stands (by reading the diff); #16166 added 'if (fromDoc >= endDoc) return;' to Lucene90's sorted-numeric overrides and open #16835 adds the same guard to its new table-encoded paths (evidence that an empty range is meant to be a no-op); open #16897 would add an in-tree caller that passes from = max(approximation.docID(), offset) without checking from < upTo (a possible future reach, not executed). Same-day recheck: no PR or issue created or updated since 2026-11-01 reports it; #16917, #16897 and #16835 are still open; no upstream commit touched PanamaDocValuesRangeSupport.java after 42e71d8e73f.
Lucene commit tested: main 5adff78bb168 (2026-11-02) through gradle in a scratch worktree, with assertions on and off, without and with the fix (the repro, the Lucene90 doc-values format suites, the doc-values range-iterator suites, TestVectorizationProvider, checkGoogleJavaFormat): with -Ptests.defaultvectorization=true the Panama provider ran; with Lucene's local default test options the scalar provider ran. Also through a standalone JUnit runner on the main jars. Equivalent tests ran on 10.5.0, 10.5.2 and 10.6.0 (fail) and 10.4.0 and 9.12.3 (no API). Only Linux aarch64 with OpenJDK 25.0.3 (256-bit and 128-bit vectors) was run; x86 was not. An independent fact check re-ran the repro on main and probed the single-valued sorted-numeric and dense sorted paths: both mark doc 4 for (12, 5) (assertions on and off) and pass with the fix.

Lucene and upstream sources

Finding record: lucene/core/src/java25/org/apache/lucene/internal/vectorization/PanamaDocValuesRangeSupport.java:48 (loopBound), :51 (SIMD loop), :65-70 (scalar tail); no fromDoc >= toDoc guard in the method
lucene/core/src/java/org/apache/lucene/index/NumericDocValues.java:113-140 (javadoc '[fromDoc, toDoc)', fromDoc inclusive, toDoc exclusive, no ordering precondition; default loop 'for (int d = fromDoc; d < toDoc; d++)')
lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java:533-543 (static forwarding to the vectorization provider), :1019-1030 (dense direct override), :1066-1076 (gcd/delta dense override), :1548-1555 (dense SORTED ordinalRangeIntoBitSet), :2079-2087 ('int endDoc = Math.min(toDoc, maxDoc); if (fromDoc >= endDoc) return;' in the sorted-numeric override)
lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultDocValuesRangeSupport.java:39 (scalar loop, no-op for a reversed range) and DocValuesRangeSupport.java:25-45 (@lucene.internal, same half-open contract)
lucene/core/src/java/org/apache/lucene/search/DocValuesRangeIterator.java:448-453 (in-tree caller: from = Math.max(blockStart, numericValues.docID()) up to blockEnd)
lucene/core/src/java/org/apache/lucene/internal/vectorization/VectorizationProvider.java:150-153 (hint to pass --add-modules jdk.incubator.vector) and build-tools/build-infra/src/main/java/org/apache/lucene/gradle/plugins/java/TestsAndRandomizationPlugin.java:453-458 (Lucene's test JVM enables the module)
The finding in 30 seconds: lucene/core/src/java/org/apache/lucene/index/NumericDocValues.java:113-140 (at 5adff78bb168) (javadoc, default loop)
lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java:533-543, (at 5adff78bb168) :1019-1030, :1066-1076, :1549-1555, :1655-1660, :2003, :2083-2086
lucene/core/src/java/org/apache/lucene/index/SingletonSortedNumericDocValues.java:80 (at 5adff78bb168) (forwards to the wrapped NumericDocValues)
lucene/core/src/java25/org/apache/lucene/internal/vectorization/PanamaDocValuesRangeSupport.java:36-71 (at 5adff78bb168) (line 48 loopBound, :51 SIMD loop, :65-70 scalar tail), :73-116 (sortedNumericRangeIntoBitSet)
lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultDocValuesRangeSupport.java:39 (at 5adff78bb168)
JDK 25 javadoc of jdk.incubator.vector.VectorSpecies#loopBound (largest multiple of the lane count that is <= the length)
Who is affected: lucene/core/src/java/org/apache/lucene/search/DocValuesRangeIterator.java:452-453, (at 5adff78bb168) :483-484, :517-518 (the three bulk range iterators); index/SingletonSortedNumericDocValues.java:80 and codecs/lucene90/Lucene90DocValuesProducer.java:1660 (forwarders)
lucene/core/src/java/org/apache/lucene/internal/vectorization/VectorizationProvider.java:150-153, (at 5adff78bb168) :156-172
Lucene main 5adff78bb168 build-tools/build-infra/src/main/java/org/apache/lucene/gradle/plugins/java/TestsAndRandomizationPlugin.java:453-458, :629-642
Why it looks unintended: lucene/core/src/java/org/apache/lucene/index/NumericDocValues.java:113-140; (at 5adff78bb168) lucene/core/src/java/org/apache/lucene/internal/vectorization/DocValuesRangeSupport.java:33-44
lucene/core/src/java/org/apache/lucene/internal/vectorization/DefaultDocValuesRangeSupport.java:39; (at 5adff78bb168) lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java:2083-2086
lucene/core/src/java/org/apache/lucene/util/FixedBitSet.java:227-228, (at 5adff78bb168) :777-779; lucene/core/src/java/org/apache/lucene/search/DocIdSetIterator.java:47-51, :187-210
Suggested fix: lucene/core/src/java25/org/apache/lucene/internal/vectorization/PanamaDocValuesRangeSupport.java:36-48 (at 5adff78bb168)
Prior reports: github.com/apache/lucene search (2026-11-03): items updated since 2026-11-02 matching rangeIntoBitSet (#16835 only) or PanamaDocValuesRangeSupport (none); items created since 2026-11-02 (#16962-#16966, deprecation clean-ups)
History: github.com/apache/lucene commit list of PanamaDocValuesRangeSupport.java (42e71d8e73f latest; read 2026-11-03)
Diagrams: lucene/core/src/java25/org/apache/lucene/internal/vectorization/PanamaDocValuesRangeSupport.java:48, (at 5adff78bb168) :51, :65-70

Other statements trace to the hunt's working files (models, logs, reviews and briefs), which are not published.

Repro files

TestReversedRangeIntoBitSet.java (lucene/core, package org.apache.lucene.index, 1 test): repro/LFV-1145/TestReversedRangeIntoBitSet.java (48 lines)

About the diagrams

The diagram illustrates the repro's call (main 5adff78bb168, PanamaDocValuesRangeSupport.java:48, :51, :65-70).

assets/LFV-1145/bug.png (517 KB)

Built 2026-11-06 from the hunt's record of this finding and its doc file. Lucene main under test in this hunt: ea57360912ed. Every listed finding was re-run on Lucene main e1faeba08bde on 2026-11-02.