The Lucene90TermVectorsFormat javadoc describes three parts of the .tvd chunk as the opposite of what Lucene writes

The class javadoc of Lucene90TermVectorsFormat, the term-vectors format of the default codec, presents itself as "a more detailed description of the field data file format" (Lucene90TermVectorsFormat.java:80 on main 645d4fe14a20), and the default codec's file-format overview links it as the term-vector format. Three of its statements say the opposite of what Lucene90CompressingTermVectorsWriter writes and Lucene90CompressingTermVectorsReader reads: the Flags Bit is documented as 1 for "fields have the same options" (:90-92, :125-126) but the writer writes 0 for that and 1 otherwise (writer :519-534); FieldNums are documented as "a delta-encoded list" (:89, :119-120) but are the plain sorted field numbers (writer :466-478); and AvgCharsPerTerm is documented as absent "if no field has both positions and offsets enabled" (:143-144) but is written whenever any field of the chunk has offsets (writer :629, :643-657). In normal indexing the reader matches the writer on all three, so the wrong text breaks nothing at runtime (their one known disagreement, on the AvgCharsPerTerm condition, is the separate code bug LFV-244). The text came with LUCENE-4733 in 2013 for a writer that already did all three things the way it does today, and it has been in every release from 4.2.0 to 10.6.0 and on main.

One document, field f = "x", term vectors without positions or offsets, decoded with the reader's rules (_0.tvd is 84 bytes):

bytes 0-48   IndexHeader (codec "Lucene90TermVectorsData")
byte  49     DocBase 0
byte  50     ChunkDocs VInt 3 = (1 doc << 1) | dirty bit 1
byte  51     NumFields 1
byte  52     FieldNums token 0x01 (1 distinct field, 1 bit per field number)
byte  53     FieldNums: 00 (field number 0)
bytes 54-55  FieldNumOffs slice: length 1, 00
byte  56     Flags Bit = 0      <- the javadoc says 1 for "same options"
bytes 57-58  flags slice: length 1, 00 (one Flag per distinct field)

• FieldNums: term-vector fields with field numbers 1, 3 and 6 are stored as [1, 3, 6]; delta-decoding as the javadoc says gives [1, 4, 10].
• AvgCharsPerTerm: a document with three fields that have offsets but no positions (x at 3-4, y at 5-6, z at 7-8). The javadoc says no AvgCharsPerTerm values are present; the file holds three 4-byte values. Decoding without them gives start offsets [-1, -1, -1]; skipping them gives the indexed [3, 5, 7].

• flushFlags sets nonChangingFlags when every field number has the same flags in every document of the chunk (writer :502-517), then writes writeVInt(0) with the comment "write one flag per field num" (:520-531), else writeVInt(1), "write one flag for every field instance" (:532-545). The reader's case 0 reads one Flag per distinct field and expands it per field instance (:447-458); case 1 reads one per instance (:459-461).
• flushFieldNums writes a token byte (min(n - 1, 7) << 5) | bitsRequired(max) (:466-468), a VInt when n - 1 >= 7 (:469-471), then writer.add(fieldNum) for each sorted distinct field number (:472-478). The reader assigns fieldNums[i] = (int) it.next() (:435).
• flushOffsets sets hasOffsets |= fd.hasOffsets over every field instance (:629), returns only when no field has offsets (:643-646), and writes one Float.floatToRawIntBits(charsPerTerm[i]) per distinct field (:655-657). The reader reads them when TotalOffsets, the sum of term frequencies of fields with offsets, is > 0 (:553-566, :589-594). In normal indexing a term-vector field with no terms is not written (TermVectorsConsumerPerField.java:70), so the two conditions agree; they are shown to differ for a zero-term field from addIndexes(CodecReader...), the separate code bug LFV-244 (other ways to hand the writer a zero-term offsets field were not ruled out exhaustively).

• Who is hit. Anyone who decodes a .tvd file without Lucene, or checks a codec change against the description, by following the class javadoc. The default codec's overview (lucene104/package-info.java) says it "defines the index file formats used in this version of Lucene" (:50), links this class for term vectors (:168, :293, :298) and says "Details on each of these are provided in their linked pages" (:188).
• Default vs opt-in. The default codec writes this format (Lucene104Codec.java:71); term vectors are opt-in per field. Which chunks contradict which statement: Flags, every chunk that holds term-vector fields (a chunk with none writes no Flags, writer :401-403); FieldNums, any chunk with two or more distinct term-vector field numbers above 0 (with one field, or field 0 and one other, the plain list and its delta encoding are the same bytes, as in the minimal repro's chunk); AvgCharsPerTerm, any chunk where some field has offsets and no field has both positions and offsets, e.g. one whose term-vector fields have offsets but no positions, a documented option that BaseTermVectorsFormatTestCase exercises (Options.OFFSETS).
• Scale. The wrong text itself has no runtime effect. A parser that follows the text reads the Flags with the wrong layout, sums plain field numbers as deltas, and loses alignment on chunks with offsets but no positions (all executed). The audience is small and no earlier report was found in the recorded searches, which keeps the impact minor.
• Rating. Impact class minor, likelihood plausible, priority low.

The default codec is pinned, compound files are off, and the test builds a plain IndexWriterConfig and a ByteBuffersDirectory rather than the framework's randomized ones, so the bytes under test are fixed by the test itself; the test framework still randomizes unrelated state such as the locale and time zone, and the commands fix the seed. Besides the test framework, the test uses the @lucene.internal PackedInts helper, only to compute how many bytes to skip. The three readVInt() calls skip DocBase, ChunkDocs and NumFields; the next lines skip the FieldNums token and packed field numbers, then the length-prefixed FieldNumOffs slice. The codec name Lucene90TermVectorsData is the one the format passes to its superclass (Lucene90TermVectorsFormat.java:170).

The other two statements (FieldNums, AvgCharsPerTerm) are shown by the earlier, longer tests of the finder and the independent reproducer, which fail on main 645d4fe14a20 with assertions on and off; they are left out of the repro on purpose.

On 9.0.0 the test-framework imports are org.apache.lucene.util.LuceneTestCase and org.apache.lucene.util.TestUtil (moved to org.apache.lucene.tests.* in 9.1.0).

As a regression test: none is needed for a javadoc-only fix, and this test cannot guard the javadoc (it transcribes the text). If a reviewer wants the corrected statement pinned down, it is this same test with expected value 0.

cp TestTermVectorsFormatJavadocFlagsBit.java $LUCENE/lucene/core/src/test/org/apache/lucene/codecs/lucene90/
cd $LUCENE && ./gradlew :lucene:core:test --tests org.apache.lucene.codecs.lucene90.TestTermVectorsFormatJavadocFlagsBit -Ptests.seed=DEADBEEF
# assertions off: add -Ptests.asserts=false (same result)

TestTermVectorsFormatJavadocFlagsBit > testSameOptionsChunkHasFlagsBitOne FAILED
    java.lang.AssertionError: Flags Bit of a one-document, one-field chunk expected:<1> but was:<0>
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:CCC0AE82C6CD4BB0]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.failNotEquals(Assert.java:835)
        at org.junit.Assert.assertEquals(Assert.java:647)
        at org.apache.lucene.codecs.lucene90.TestTermVectorsFormatJavadocFlagsBit.testSameOptionsChunkHasFlagsBitOne(TestTermVectorsFormatJavadocFlagsBit.java:60)

1 test completed, 1 failed

• Inverted, not simplified. if Bit==1: Flag^TotalDistinctFields and "when true means that fields have the same options" (:91-92, :125-126) state the opposite of the writer, whose comments read "write one flag per field num" for 0 (:520) and "write one flag for every field instance" for 1 (:533).
• Wrong from the start. LUCENE-4599 (5837fe83bd8, 2013-01-21) wrote the writer; LUCENE-4733 (94b17a28de5 / e3710747d87, 2013-02-02) added the description; the writer file is byte-identical between the two, so the text never described a committed encoding. Not a regression.
• The page presents itself as the detailed description: "Here is a more detailed description of the field data file format" (:80); the default codec's overview sends readers there for the details (package-info.java:50, :188).
• Precedent. LUCENE-713 (#1788, 2006, "File Formats Documentation is not correct for Term Vectors") fixed an error in the old term-vector format docs; #129 (2021) re-synced this javadoc with a format change. No test, CHANGES or MIGRATE entry or discussion calls these statements known or non-normative.

Strongest case for "intended" (the devil's-advocate verifier and the cross-model reviewers): the overview "attempts to provide a high-level definition of the Apache Lucene file formats" (package-info.java:54); the class is @lucene.experimental (Lucene90TermVectorsFormat.java:164); the description is incomplete anyway (no FieldNums token byte, no DirectWriter slice length prefixes, no dirty bit in ChunkDocs), so nobody could write a reader from it alone; the author of #130 remarked that "few people care about the format documentation"; and in normal indexing the writer and reader agree. The refute-lens reviewer rated this moderate, for the encoding only.

Why it fails: "high-level" covers leaving details out, not a statement whose polarity is inverted. @lucene.experimental permits incompatible future changes, "not inaccurate documentation of the current release" (the intended-lens reviewer, who rated the argument weak). "Few people care" explains why nobody noticed; it does not make the text right. Writer/reader agreement in normal indexing is why this is a documentation defect, not a code bug (their one known disagreement, LFV-244, is a separate code bug). The incompleteness argument does shape the claim: it is only that these three statements are wrong, not that a reader built from the javadoc must fail. The reproducer, the devil's advocate and the adjudication all kept the defect, and the separate re-runs of the finder's test for the cross-model reviews failed.

One option was written, built and tested (no test reads the javadoc); a wider cleanup is listed as option B. The AvgCharsPerTerm sentence depends on LFV-244, the code bug on the same condition: if its fix makes the writer follow the reader, the sentence should say "present when TotalOffsets is greater than 0" instead (in normal indexing both say the same). Open PR #16480 (per-chunk checksums) changes the term-vectors writer and reader but not this javadoc; open PR #1003 touches only the file's import and constructor.

Javadoc only, one file, 7 lines in and 7 out: FieldNums --> FieldNum^TotalDistinctFields; FieldFlags --> if Bit==0: ...; "FieldNums is the list of the sorted unique field numbers present in the chunk"; "Bit in Flags is 0 when fields have the same options for every document in the chunk, and 1 otherwise"; "They are present if any field of the chunk has offsets enabled, and are 0 for a field that does not have both positions and offsets". The new text matches what normal indexing writes and reads and the writer's presence condition; on the LFV-244 path the writer and reader disagree on that condition. The last clause states when the value is certainly 0, not every case: the writer also writes 0 when a field's sum of positions or of start offsets is 0 (writer :648-652, e.g. one token at position 0, start offset 0); it agrees with the existing StartOffsetDelta note ("AvgCharsPerTerm is 0 if the field has no positions"), a sufficient condition in the same way. If a reviewer wants the exact rule, drop the clause or say "for example".

diff --git a/lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90TermVectorsFormat.java b/lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90TermVectorsFormat.java
index e19168ff95d..d45a9fbfda2 100644
--- a/lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90TermVectorsFormat.java
+++ b/lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90TermVectorsFormat.java
@@ -86,9 +86,9 @@ import org.apache.lucene.util.packed.PackedInts;
  *             TermFreqs >, < Positions >, < StartOffsets >, < Lengths >, <
  *             PayloadLengths >, < TermAndPayloads >
  *         <li>NumFields --&gt; DocNumFields<sup>ChunkDocs</sup>
- *         <li>FieldNums --&gt; FieldNumDelta<sup>TotalDistincFields</sup>
+ *         <li>FieldNums --&gt; FieldNum<sup>TotalDistinctFields</sup>
  *         <li>Flags --&gt; Bit &lt; FieldFlags &gt;
- *         <li>FieldFlags --&gt; if Bit==1: Flag<sup>TotalDistinctFields</sup> else
+ *         <li>FieldFlags --&gt; if Bit==0: Flag<sup>TotalDistinctFields</sup> else
  *             Flag<sup>TotalFields</sup>
  *         <li>NumTerms --&gt; FieldNumTerms<sup>TotalFields</sup>
  *         <li>TermLengths --&gt; PrefixLength<sup>TotalTerms</sup>
@@ -116,14 +116,13 @@ import org.apache.lucene.util.packed.PackedInts;
  *         <li>DocBase is the ID of the first doc of the chunk.
  *         <li>ChunkDocs is the number of documents in the chunk.
  *         <li>DocNumFields is the number of fields for each doc.
- *         <li>FieldNums is a delta-encoded list of the sorted unique field numbers present in the
- *             chunk.
+ *         <li>FieldNums is the list of the sorted unique field numbers present in the chunk.
  *         <li>FieldNumOffs is the array of FieldNumOff; array size is the total number of fields in
  *             the chunk.
  *         <li>FieldNumOff is the offset of the field number in FieldNums.
  *         <li>TotalFields is the total number of fields (sum of the values of NumFields).
- *         <li>Bit in Flags is a single bit which when true means that fields have the same options
- *             for every document in the chunk.
+ *         <li>Bit in Flags is 0 when fields have the same options for every document in the chunk,
+ *             and 1 otherwise.
  *         <li>Flag: a 3-bits int where:
  *             <ul>
  *               <li>the first bit means that the field has positions
@@ -141,7 +140,8 @@ import org.apache.lucene.util.packed.PackedInts;
  *             difference with the previous positions for following positions.
  *         <li>TotalOffsets is the sum of frequencies of terms of all fields that have offsets.
  *         <li>AvgCharsPerTerm is the average number of chars per term, encoded as a float on 4
- *             bytes. They are not present if no field has both positions and offsets enabled.
+ *             bytes. They are present if any field of the chunk has offsets enabled, and are 0 for
+ *             a field that does not have both positions and offsets.
  *         <li>StartOffsetDelta is the (startOffset - previousStartOffset - AvgCharsPerTerm *
  *             PositionDelta). previousStartOffset is 0 for the first offset and AvgCharsPerTerm is
  *             0 if the field has no positions.

Trade-offs

None at runtime: no bytes, behaviour or format version change. It corrects only these three statements and is not a complete correction of the .tvd description: the other stale lines (option B) stay as they are.

What the checks validated

Scratch worktree of 645d4fe14a20, seed DEADBEEF, 2026-11-01, removed afterwards:

• stock: the minimal repro fails with assertions on and off;
• with the fix: the same test with expected value 0 passes with assertions on and off; the verbatim repro still fails with the same message, as it must (the fix changes no bytes);
• TestLucene90TermVectorsFormat: 21 tests, 0 failures;
• :lucene:core:check -x test (google-java-format, ecj lint, forbidden APIs) succeeds with only the javadoc file changed.
• no test reads the javadoc, so these runs show only that the edit is clean and changes no bytes; that the new text is right is a code-reading conclusion; the javadoc was not rendered to HTML (renderJavadoc not run).

ChunkDocs is written as (docs << 1) | dirtyBit since LUCENE-9935 (#140; a 2-document chunk stores 5); FieldNumOffs, Flags and FieldNumTerms are DirectWriter slices with a length prefix since LUCENE-9907 (#72), not PackedInts arrays or "blocks of 64 packed ints"; FieldNums is preceded by a token byte (min(n - 1, 7) << 5) | bitsRequired(max) (plus a VInt when n - 1 >= 7); "DocNumFields (with ChunkDocs>=1)" should say >1 (typo from #130); Bit is written as a VInt; AvgCharsPerTerm is listed as an Int but holds raw float bits.

Trade-offs

A byte-accurate description, but a larger diff for reviewers to check against more code. Not written as a diff and not built or tested here.

It keeps three sentences that state the opposite of the bytes.

Trade-offs

No work, but the page would still mislead the people it is written for.

• The hunt unit modelled the term-vector offsets encoding of the default format in Z3 and listed, for each modelled step, the code line it came from. Reading the javadoc next to the writer for that table showed the three statements.
• The finder's JUnit indexes one document with three fields that have offsets but no positions (one term each), parses the single .tvd chunk byte by byte and checks all three statements; its implicit control is that skipping the AvgCharsPerTerm values decodes the indexed start offsets [3, 5, 7]. It failed on main with assertions on and off, on 10.5.2 and on 9.12.3.
• This is a documentation check against real output, not model checking or a proof: the formal model gave no signal on the javadoc.