addIndexes(CodecReader) from a reader whose term vectors list a field with zero terms writes a term-vector chunk that the default codec cannot read back, and every later merge that re-encodes the segment closes the IndexWriter

Finding LFV-244Area: core / codecs: term vectors (TermVectorsWriter.addAllDocVectors merge copier; Lucene90 compressing term-vector writer and reader), reached through IndexWriter.addIndexes(CodecReader)Priority: lowImpact: data corruptionNovelty: new

Finding page for LFV-244

AI-assisted finding, confirmed by automated checks, not yet triaged by a human Lucene committer. This site files nothing upstream.

What it shows

You copy an index with IndexWriter.addIndexes(CodecReader...) through a reader that removes some terms consistently from the postings and the term vectors (CheckIndex at its strictest level accepts this reader), so one document's term-vector field is left with zero terms. You expect the new segment to read back and merge like any other, as the postings do (the postings writer simply drops the emptied field). Instead the default term-vector format writes per-field AvgCharsPerTerm floats that its own reader does not expect, so when the emptied field stores offsets the other documents of the chunk throw 'offset 0 is invalid' on read, and because the reader lists the emptied field but returns null for it, the next merge that re-encodes the segment fails with 'Wrong number of values added' and closes the IndexWriter, on every retry.

The test shows that addIndexes(CodecReader) from a reader whose term vectors list a field with zero terms (but offsets) writes a term-vector chunk that the default codec cannot read back, which damages another document of the chunk.

Expected vs actual

Expected: doc 1's term vector for field b reads back as y. Actual: expected:<[y]> but was:<[java.io.IOException: offset 0 is invalid]>.

Files

• src/org/apache/lucene/index/TestTermVectorsZeroTermField.java (JUnit test)

Run it

• Tested on Lucene main e1faeba08bde (2026-11-02): still reproduces.
• Tested on Lucene main 645d4fe14a20 (2026-10-28): reproduced.
• Recorded outcome of run 1 on Lucene main 645d4fe14a20: 1 test(s) run, 1 failed.
• How to read the outcome: a failing test is the finding reproducing (the failure message states expected against actual).
• Seeds: a rerun with the same seed replays the same random choices. Assertions off (-da -dsa -Dtests.asserts=false) is the production-like setting.

# Lucene checkout at the tested commit, built with: ./gradlew --no-daemon assemble   (JDK 25)
LUCENE=/path/to/lucene; V=12.0.0-SNAPSHOT
CP=$LUCENE/lucene/core/build/libs/lucene-core-$V.jar
CP=$CP:$LUCENE/lucene/codecs/build/libs/lucene-codecs-$V.jar
CP=$CP:$LUCENE/lucene/test-framework/build/libs/lucene-test-framework-$V.jar
CP=$CP:randomizedtesting-runner-2.9.2.jar:junit-4.13.2.jar:hamcrest-3.0.jar   # test-framework dependencies (Maven Central)
CP=$CP:junit-platform-console-standalone-6.1.3.jar   # JUnit Platform console launcher (Maven Central)

javac -d classes -cp "$CP" $(find src -name '*.java')

# run 1: assertions on, seed DEADBEEF
java -ea -esa --add-modules jdk.incubator.vector --enable-native-access=ALL-UNNAMED -Dtests.seed=DEADBEEF \
     -cp "classes:$CP" org.junit.platform.console.ConsoleLauncher \
     execute --select-class org.apache.lucene.index.TestTermVectorsZeroTermField

Gradle alternative: copy the test into the matching module's src/test/ tree of the checkout and run ./gradlew -p lucene/<module> test --tests <class> -Ptests.seed=<seed> (add -Ptests.asserts=false for a run without assertions).

Formal models and scripts this finding's part of the hunt used: termvectors-compressing

Feedback

Open an issue in lucene-ai-bug-hunt. Please do not open issues or pull requests in apache/lucene about this finding unless a Lucene committer agrees it should be filed.

Snapshot of 2026-11-04. AI-assisted; not yet triaged by a human Lucene committer. Feedback: open an issue in lucene-ai-bug-hunt.