The default SortedSetDocValues terms enum casts the ordinal to int in seekExact(long): a seek to an ordinal of 2^31 or more throws or silently lands on the wrong term

SortedSetDocValues#termsEnum() returns, by default, a SortedSetDocValuesTermsEnum and documents that "The enum supports TermsEnum#ord() and TermsEnum#seekExact(long)". Sorted-set ordinals are long, and the enum keeps its position in a long field, but seekExact(long ord) stores currentOrd = (int) ord; before looking the value up. The cast keeps the low 32 bits as a signed int, and the assert one line above checks the un-narrowed long, so assertions do not catch it.

On a view with more than 2^31 distinct values a seek to an ordinal from 2^31 to 2^32 - 1 makes the enum look up a negative ordinal, which Lucene's views reject with an exception; from 2^32 to 2^32 + 2^31 - 1 it silently lands on ord mod 2^32: seekExact(4294967301) leaves the enum on ordinal 5 with the term of ordinal 5. Higher up the two behaviours alternate in bands of 2^31. seekExact(term, TermState) delegates to the same method. Only views that keep the default enum are affected: the top-level MultiDocValues view over several segments, SortingCodecReader's sorted-set view (documented for merging only), SimpleText's multi-valued fields, the test framework's asserting wrapper and application subclasses; per-segment values of the default codec have their own correct enum, and no Lucene code seeks these views with an ordinal of 2^31 or more. The line dates from 2013 (4.2.0) and is unchanged on main; currentOrd = ord; fixes it.

// SortedSetDocValuesTermsEnum (abridged)
private long currentOrd = -1;                                 // :26

public void seekExact(long ord) throws IOException {          // :67
  assert ord >= 0 && ord < values.getValueCount();            // :68, checks the long
  currentOrd = (int) ord;                                     // :69, keeps the low 32 bits
  scratch.copyBytes(values.lookupOrd(currentOrd));            // :70
}

public void seekExact(BytesRef term, TermState state) throws IOException {  // :114
  this.seekExact(((OrdTermState) state).ord);                 // :116, same cast
}

In general an ordinal whose low 32 bits read as a negative int (2^31 to 2^32 - 1, 3 x 2^31 to 2^33 - 1, ...) makes the enum look up a negative ordinal. Lucene's own views reject that with an exception (by code reading; the type depends on the view: IndexOutOfBoundsException from the default codec's bounds check, IllegalArgumentException from the finder's synthetic values), and an application subclass's lookupOrd decides for itself. Any other ordinal of 2^32 or more (2^32 to 2^32 + 2^31 - 1, 2^33 to 2^33 + 2^31 - 1, ...) lands on ord mod 2^32. After a bad seek, ord(), term(), termState() and next() all continue from the wrong position.

The class was created in 2013 as a copy of SortedDocValuesTermsEnum, which has private int currentOrd and the same line; there it is correct, because SortedDocValues ordinals are ints. The copy widened the field to long and kept the line. That it is a copy leftover is an inference: no comment, test or commit message explains it.

• MultiDocValues.MultiSortedSetDocValues (@lucene.internal), returned by MultiDocValues.getSortedSetValues for a reader with more than one segment when at least one segment has the field (otherwise it returns null); its ordinals are global ordinals from an OrdinalMap, so the bound is on distinct values across all segments. Executed by an independent check over a real OrdinalMap with two synthetic segments: fails, passes with the fix.
• SortedSetDocValuesWriter.SortingSortedSetDocValues, returned by SortingCodecReader's sorted-set producer. SortingCodecReader's javadoc says it "should only be used for merging", and merging walks these values with next().
• SimpleText's multi-valued sorted-set values (a debugging codec; single-valued SimpleText fields come back as a singleton and use the sorted enum), the test framework's AssertingLeafReader.AssertingSortedSetDocValues wrapper, the unused debugging helper GenericTermsCollector.verbose in join, and any application or downstream SortedSetDocValues subclass that does not override termsEnum(). FilterSortedSetDocValues and SingletonSortedSetDocValues delegate to the wrapped values.
• Not affected: per-segment sorted-set values of the default codec, which return their own terms dictionary enum; its seekExact(long) works on the long. The merged view built for sorted-set merges also brings its own enum (MergedTermsEnum).

• Not Lucene's own code. Merges only iterate these enums with next(); the cited facet and join classes reject more than Integer.MAX_VALUE values up front; grouping's term facets seek SortedSetDocValues#termsEnum() by ordinal, but narrow the value count to int and seek with int ordinals, so they cannot reach 2^31. So no Lucene query, filter, facet, join, grouping or merge returns a wrong result from this (code reading; tests seek these enums only with small ordinals).
• Default, not opt-in. No setting is involved: the enum is the default of the public base class. Assertions on and off behave the same.
• Scale. 2^31 = 2,147,483,648 distinct values in one view, 2^32 = 4,294,967,296 for the silent case. The format allows it (the default codec writes the value count as a variable-length long), and a maintainer stated on #11636 / LUCENE-10600 that SORTED_SET fields can exceed Integer.MAX_VALUE unique values. For a multi-segment view the bound is on distinct values across all segments.
• Effect. Ordinals 2^31 to 2^32 - 1, and every later band whose low 32 bits read as a negative int: the enum asks lookupOrd for a negative ordinal, which Lucene's own views reject with an exception (code reading; every executed view threw); an application subclass decides for itself. Ordinals 2^32 to 2^32 + 2^31 - 1, and every later band whose low 32 bits are non-negative: the enum silently reports ordinal ord mod 2^32 and its term, and next() continues from there. No memory unsafety (Java bounds checks).
• Likelihood: contrived. It needs billions of distinct values in one view plus an ordinal seek on the default enum; nobody has reported it in 13 years.

• Why this case. It is the worst effect, the silent wrong position, in one assertion. The exception between 2^31 and 2^32 - 1 and the seekExact(term, TermState) overload are the same line and are fixed by the same token.
• The strict reading, "only ordinals previously returned by ord()". The bare seekExact(ord) does not first get the ordinal from ord(), but it is exactly what ord() reports for that term: on the same values, te.seekCeil(values.lookupOrd(ord)) returns FOUND with ord() 4,294,967,301, and seekExact with that ord() value, or with te.termState() after moving away, gives ord() 5 (both executed).
• Releases. The same file compiles and fails unchanged on 10.6.0.
• As the regression test: this same test, for example as a method of TestMultiDocValues or in a new TestSortedSetDocValuesTermsEnum. It allocates nothing big, so it needs no @Monster annotation. It passes with the fix.
• Classes used. Nothing package-private; the term encoding uses BitUtil.VH_BE_LONG, and BitUtil is public but marked @lucene.internal.

cp $REPRO/TestSortedSetTermsEnumSeekExactLongOrd.java $LUCENE/lucene/core/src/test/org/apache/lucene/index/
cd $LUCENE
./gradlew :lucene:core:test --tests org.apache.lucene.index.TestSortedSetTermsEnumSeekExactLongOrd -Ptests.seed=DEADBEEF
# assertions off: add -Ptests.asserts=false (same result)

# gradle, main 645d4fe14a20, seed DEADBEEF, assertions on (the same with -Ptests.asserts=false)
TestSortedSetTermsEnumSeekExactLongOrd > testSeekExactOrdAbove2Pow32 FAILED
    java.lang.AssertionError: ord() after seekExact(4294967301) expected:<4294967301> but was:<5>
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:5F7086835E0F3B8A]:0)
        at org.junit.Assert.assertEquals(Assert.java:647)
        at org.apache.lucene.index.TestSortedSetTermsEnumSeekExactLongOrd.testSeekExactOrdAbove2Pow32(TestSortedSetTermsEnumSeekExactLongOrd.java:81)

# 10.6.0 release jar, assertions on and off: the same AssertionError

# the exception band, from the finder's earlier test (synthetic values that reject negative ordinals)
testSeekExactByOrd                  java.lang.IllegalArgumentException: ord out of range: -2147483641
testSeekExactByTermStateOfSameEnum  java.lang.IllegalArgumentException: ord out of range: -2147483641

• Contract. SortedSetDocValues#termsEnum(): "Returns a TermsEnum over the values. The enum supports TermsEnum#ord() and TermsEnum#seekExact(long)." Ordinals and getValueCount() are long ("one plus the maximum ordinal"). TermsEnum#seekExact(long): "Seeks to the specified term by ordinal (position) as previously returned by ord(). The target ord may be before or after the current ord, and must be within bounds." seekExact(BytesRef, TermState) is legal with a state from the same enum instance.
• The class is long-ordinal everywhere else. seekCeil, seekExact(BytesRef), next(), ord() and termState() use the long field, and OrdTermState.ord is a long. The cast is the only narrowing.
• Upstream tests exercise this method. LegacyBaseDocValuesFormatTestCase.testSortedSetTermsEnum and testSortedSetEnumAdvanceIndependently (through its helper) seek by ordinal on multi-valued documents, and they run through TestSimpleTextDocValuesFormat, whose multi-valued sorted-set view uses this enum. Ordinal seeking is supported, just never tested above 2^31.
• Precedent. rmuir's commit 7ab10178c38 (LUCENE-5767, 2014): "remove bogus cast (in this case can exceed Integer.MAX_VALUE, and the underlying delta reader takes long anyway)", changing (int) ord to ord in the global-ordinals code of MultiDocValues.
• No documented limit. No javadoc, test, comment or CHANGES entry limits sorted-set views to 2^31 values; the facet and join guards are module-local caps for their own int arrays.

The strongest case for "works as designed" (built by the intended-behaviour checker): (a) the enum is a package-private, generic fallback that LUCENE-4819 put behind termsEnum(), which codecs can override, and the default codec does; (b) the views that keep it are niche: MultiDocValues is @lucene.internal and @lucene.experimental ("This is very costly"), SortingCodecReader is documented as "should only be used for merging", and SimpleText is a debugging codec ("FOR RECREATIONAL USE ONLY"); (c) no Lucene code seeks them with an ordinal of 2^31 or more: the sorted-set merge view brings its own enum, grouping uses int ordinals, and the cited facet and join classes reject that many values; (d) seekExact(long) takes ordinals "as previously returned by ord()", which the minimal test's bare seekExact(2^32 + 5) is not; (e) no real index with more than 2^31 distinct values was built: the failing view is synthetic.

Why it fails: the public method SortedSetDocValues#termsEnum() promises seekExact(long) and every subclass that does not override it gets this enum; @lucene.internal and @lucene.experimental allow API changes, not a long-ordinal API that narrows to int, and niche, merge-only or debugging views limit reach, not correctness. The strict reading is answered by the round trips: seekCeil returns FOUND with ord() 4,294,967,301, and seeking back with that ord() or with that enum's termState() lands on 5. A synthetic view is the practical way to reach 2^31 values in a test; it uses only public classes, and an independent check reached the same line through a real OrdinalMap and MultiSortedSetDocValues over synthetic segments. "Nobody calls it at this scale" limits severity, not validity. Verdict: not intended, a defect; low priority.

One fix was built and validated; the alternatives are noted with why they are worse.

Store the ordinal as the long it is. Covers seekExact(long) and seekExact(term, TermState).

--- a/lucene/core/src/java/org/apache/lucene/index/SortedSetDocValuesTermsEnum.java
+++ b/lucene/core/src/java/org/apache/lucene/index/SortedSetDocValuesTermsEnum.java
@@ -66,7 +66,7 @@
   @Override
   public void seekExact(long ord) throws IOException {
     assert ord >= 0 && ord < values.getValueCount();
-    currentOrd = (int) ord;
+    currentOrd = ord;
     scratch.copyBytes(values.lookupOrd(currentOrd));
   }
 

Trade-offs

None found: below 2^31 the behaviour is identical, and it removes an instruction. The sibling SortedDocValuesTermsEnum keeps its cast, which is correct for int ordinals.

What the checks validated

• Scratch worktree of main 645d4fe14a20 with the diff applied, gradle, OpenJDK 25.0.3 (2026-11-01): the minimal test passes with assertions on and off.
• Closest core suites, seed DEADBEEF (TestMultiDocValues, TestOrdinalMap, TestSortingCodecReader, TestDocValues, TestLucene90DocValuesFormat*): 7 classes, 526 tests, 55 skipped, 0 failures.
• TestSimpleTextDocValuesFormat in :lucene:codecs, whose sorted-set tests seek this very enum by ordinal: 148 tests, 10 skipped, 0 failures.
• :lucene:core:check -x test (formatting, ecj lint, forbidden APIs): passes.
• Earlier, standalone: the minimal test on main 645d4fe14a20; the finder's 3 tests, an independent reproducer's 5 tests and its OrdinalMap test on ea57360912ed; the 4.2.0 class with only the cast removed. All pass.

Throw instead of narrowing.

Trade-offs

Turns the silent wrong position into an exception, but invents a 2^31 limit the long-ordinal API does not have and still rejects valid seeks. Not recommended; not validated.

Give MultiSortedSetDocValues and SortingSortedSetDocValues their own enums.

Trade-offs

Leaves every application or downstream subclass that relies on the default enum broken, and is far more code than one token. Not recommended; not validated.

• The hunt unit checked that terms and postings cursors stay coherent after every move. It modelled the doc-values terms-enum views for a bounded-exhaustive translation-validation harness, with at most 4 values per view, so no tool could reach 2^31; the signal came from reading the types while writing the model.
• The finder then wrote a public-API JUnit over a synthetic SortedSetDocValues with 5 x 2^30 values that keeps the default termsEnum(): 3 tests (a seek to 2^31 + 7, a seek to 2^32 + 5, a same-instance TermState round trip), all failing on main, 10.5.2 and 9.12.3 with assertions on and off, all passing with the cast removed.
• An independent reviewer from another model family (OpenAI GPT-5.6) pointed out that from 2^32 the cast wraps to a non-negative wrong ordinal; that became the silent-case test, and later the one-method minimal repro.