The SegmentInfos javadoc describes a segments_N layout that Lucene has not written since 4.9.0

Every Lucene commit writes a small file, segments_N, that lists the index's segments, and the SegmentInfos class javadoc documents it field by field; the default codec's file-formats overview links that page as the Segments File. The documented layout no longer matches the bytes: the writer adds IndexCreatedVersionMajor (a vInt right after LuceneVersion, since 7.0.0), and per segment SoftDelCount (7.4.0), a segment-commit id (a flag byte plus 16 bytes, 8.6.0) and FieldInfosFiles (4.9.0), none of them listed; NameCounter is written as a vLong (7.2.0), not the documented Int32; and the fixed-width integers are written big-endian while the text links DataOutput#writeInt / #writeLong, which write little-endian since 9.0.0. SegID is also called the codec's identifier; it is the segment's id. Lucene's writer and reader agree, so nothing goes wrong at runtime; a reader that follows the text misreads the field right after LuceneVersion of every commit written since 7.0 (Version = 11 instead of 2 for an empty commit on main) and is out of sync from there on. The fix is a javadoc edit.

The javadoc's sequence (SegmentInfos.java:63-66): Header, LuceneVersion, Version, NameCounter, SegCount, MinSegmentLuceneVersion, <SegName, SegID, SegCodec, DelGen, DeletionCount, FieldInfosGen, DocValuesGen, UpdatesFiles, DocValuesOverlays>^SegCount, CommitUserData, Footer.

What write() writes (SegmentInfos.java:633-751; readCommit mirrors each step):

Header, LuceneVersion, MinSegmentLuceneVersion, SegName, SegCodec, CommitUserData and Footer match.

new IndexWriter(dir, new IndexWriterConfig()).close() on main writes segments_1, 69 bytes. Between the 35-byte header and the 16-byte footer (which starts C0 28 93 E8) are 18 bytes:

0B 00 00                  LuceneVersion 11.0.0 (three vInts)
0B                        IndexCreatedVersionMajor 11 (vInt, undocumented)
00 00 00 00 00 00 00 02   Version 2 (big-endian Int64)
00                        NameCounter 0 (vLong)
00 00 00 00               SegCount 0 (big-endian Int32)
00                        CommitUserData, empty map

Read as documented, Version takes 0B 00 00 00 00 00 00 00 = 11 (little-endian), NameCounter 02 00 00 00 = 2, and SegCount 00 00 00 C0 = -1073741824, already one byte into the footer.

Fixing only the commit-level items is not enough. On an independent verifier's index (3 segments, NameCounter 160, a hard delete, a soft delete, a doc-values update, user data), applying IndexCreatedVersionMajor, big-endian integers and the vLong counter one by one, the read still fails in the first segment entry at UpdatesFiles until SoftDelCount, the segment-commit id and FieldInfosFiles are added; only the full writer layout parses exactly to the footer.

• Who. People who read, inspect or re-implement segments_N from the javadoc: ports to other languages, index inspectors, recovery or forensic tools, format tooling. The default codec's overview sends them there: "This document defines the index file formats used in this version of Lucene" (lucene104/package-info.java:50), "Details on each of these are provided in their linked pages" (:188), and the file table links SegmentInfos as the Segments File (:221-222).
• Default vs opt-in. No setting is involved. Every commit written since 7.0 has the IndexCreatedVersionMajor byte where the text puts Version; every commit since 4.9.0 has FieldInfosFiles in each segment entry.
• What they lose. A reader built from the text is out of sync from the field after LuceneVersion on; they have to read write() to find the real layout.
• Scale. Small audience; the searches found no report since 2014. The prior-art search found two StackOverflow questions about decoding segments_N for recovery, so people do try; neither cites the javadoc.
• Not affected. Lucene's behaviour, results and indexes. Nothing to fix in code.

The test lives in package org.apache.lucene.index like Lucene's own index tests but uses only public API (IndexWriter, SegmentInfos.readLatestCommit, Directory.openInput, CodecUtil.checkIndexHeader, IndexInput). No documents, single thread. The random inputs do not touch the bytes it checks: the header's id is random (StringHelper.randomId(), SegmentInfos.java:638) but the test skips the header, and segments_N is written by SegmentInfos itself, so the randomized test codec cannot change the outcome. It shows the first divergence a reader following the text hits; the other stale items are read off the javadoc and write() (see the table above).

Regression test (optional): this same test reading the corrected layout: IndexCreatedVersionMajor as a vInt, then Version with CodecUtil.readBELong. It passes on today's code, so it guards the bytes, not the text.

# main 645d4fe14a20, seed DEADBEEF, -ea (the -da run prints the same)
JUnit Vintage:TestSegmentsNJavadocVersionField:testFieldAfterLuceneVersionIsVersion
  => java.lang.AssertionError: Version expected:<2> but was:<11>
     __randomizedtesting.SeedInfo.seed([DEADBEEF:2DAB27313494B3D6]:0)
     org.junit.Assert.fail(Assert.java:89)
     org.junit.Assert.failNotEquals(Assert.java:835)
     org.junit.Assert.assertEquals(Assert.java:647)
     org.apache.lucene.index.TestSegmentsNJavadocVersionField.testFieldAfterLuceneVersionIsVersion(TestSegmentsNJavadocVersionField.java:44)

# published release jars, -ea, seed DEADBEEF
10.6.0: Version expected:<2> but was:<10>
10.5.2: Version expected:<2> but was:<10>
9.12.3: Version expected:<2> but was:<9>

# the same test reading the corrected layout (IndexCreatedVersionMajor vInt, then CodecUtil.readBELong)
main 645d4fe14a20, -ea and -da: 1 test, 0 failures

# gradle, scratch worktree of 645d4fe14a20, stock source
TestSegmentsNJavadocVersionField > testFieldAfterLuceneVersionIsVersion FAILED
    java.lang.AssertionError: Version expected:<2> but was:<11>

• The contract is the page. An ordered element sequence, a type and a writer link for each element, and field descriptions (SegmentInfos.java:61-117); nothing marks it approximate. The overview: "This document defines the index file formats used in this version of Lucene" (package-info.java:50), "Details on each of these are provided in their linked pages" (:188).
• Updated as the format evolved, though not every time. LUCENE-5618 (2014) added DocValuesGen and the map form of UpdatesFiles; LUCENE-5954 (2015) added LuceneVersion and MinSegmentLuceneVersion; #16418 (Jim Ferenczi, 2026-08-13) added DocValuesOverlays to the sequence, data types and descriptions. LUCENE-7756, LUCENE-7999, LUCENE-8341 and LUCENE-9324 changed the bytes without touching the text; none of their issues or reviews mentions it.
• The oldest gap was made while editing the grammar. LUCENE-5618's commit (2014-05-21) rewrote the sequence in the same diff that added writeStringSet(getFieldInfosFiles()), and did not list it.
• Precedent. LUCENE-350 (2005, "counter field in segments file is not documented in fileformats.xml"), LUCENE-3072 ("3.1 fileformats out of date") and LUCENE-7532 (a missing 6.2 file-format description) were fixed as defects. LUCENE-7532 came from a java-user@ thread ("an overview of the index format on Javadoc", 2016-10-29) in which Michael McCandless wrote "Yes please! This is a bug."

The strongest case for "not a bug", built by the refute reviewer, the intended-lens reviewer and the devil's-advocate verifier (the cross-model reviewers rated it moderate; the devil's-advocate verifier found it fails for the core claim and conceded only the byte-order item): SegmentInfos is @lucene.experimental (SegmentInfos.java:119); the overview "attempts to provide a high-level definition" (package-info.java:54, the LUCENE-2946 policy of 2012); the supported reader is SegmentInfos.readCommit; and big-endian was chosen on purpose (on PR #107 rmuir wrote "I think its fine that segments_N and codec headers just stay bigendian"), so the writeInt / writeLong links may be width labels.

Why it fails: experimental allows the format to change, not a wrong description of the current one; LUCENE-2946 itself kept detailed docs for "the simple metadata files" ("its fine (for now)", rmuir), and this grammar stayed element-level and was extended in 2026; the page exists for people who need the layout. The byte-order point is the only one that survives, so it is the secondary item: CodecUtil's own header/footer javadoc uses the same links for big-endian fields (CodecUtil.java:60, :64, :399-403). The missing fields and the NameCounter type are wrong under any reading.

Option A was validated in a scratch worktree of 645d4fe14a20; the others were not run.

In the SegmentInfos class javadoc: add IndexCreatedVersionMajor to the sequence and SoftDelCount, SegCommitID and FieldInfosFiles to the segment entry; type NameCounter as vLong; link SegCount, DeletionCount, SoftDelCount, Version, DelGen, FieldInfosGen, DocValuesGen and the UpdatesFiles / DocValuesOverlays integers to CodecUtil#writeBEInt / #writeBELong with "(big-endian)"; spell out the UpdatesFiles count; describe SegID as the segment's id and add descriptions for the new fields; drop Generation and Checksum. 1 file, +25/-12, javadoc only (google-java-format applied). The excerpt below shows the changed grammar and type lines.

 *   <li><code>segments_N</code>: Header, LuceneVersion, IndexCreatedVersionMajor, Version,
 *       NameCounter, SegCount, MinSegmentLuceneVersion, &lt;SegName, SegID, SegCodec, DelGen,
 *       DeletionCount, FieldInfosGen, DocValuesGen, SoftDelCount, SegCommitID, FieldInfosFiles,
 *       UpdatesFiles, DocValuesOverlays&gt;<sup>SegCount</sup>, CommitUserData, Footer
 ...
 *   <li>IndexCreatedVersionMajor --&gt; {@link DataOutput#writeVInt vInt}
 *   <li>NameCounter --&gt; {@link DataOutput#writeVLong vLong}
 *   <li>SegCount, DeletionCount, SoftDelCount --&gt; {@link CodecUtil#writeBEInt Int32}
 *       (big-endian)
 *   <li>Version, DelGen, FieldInfosGen, DocValuesGen --&gt; {@link CodecUtil#writeBELong Int64}
 *       (big-endian)
 *   <li>SegCommitID --&gt; one {@link DataOutput#writeByte byte}, 1 if an id follows and 0 if not,
 *       then the id as {@link DataOutput#writeByte Int8<sup>ID_LENGTH</sup>} if the byte is 1
 *   <li>FieldInfosFiles --&gt; {@link DataOutput#writeSetOfStrings(Set) Set&lt;String&gt;}

Trade-offs

The smallest change that makes the page an accurate spec again, in the existing style and with the element names readers know; SegCommitID, UpdatesCount and FieldNumber are new names, any will do. Linking CodecUtil#writeBEInt / #writeBELong (documented as "big endian order") states the byte order without a new convention. It does not prevent future drift. Optional extras the validated diff leaves out: the untouched MinSegmentLuceneVersion line still says "this is only written only if" (an older typo, SegmentInfos.java:76), and the SegID and SegCommitID rows could link DataOutput#writeBytes, which the writer calls, instead of #writeByte.

What the checks validated

Yes, in a scratch worktree of 645d4fe14a20 (removed after the runs). With the fix, checkGoogleJavaFormat (after one formatter re-wrap), renderJavadoc (javadoc with -Xdoclint:all,-missing, which fails on an unresolved link) and ecjLintMain of core pass; every new link target is a public declaration. TestSegmentInfos (8 tests) and the minimal test rewritten to read the corrected layout pass with gradle, assertions on and off (run with the hand-written diff, which differs from the formatted one only in the wrapping of three javadoc lines). An independent verifier's parser that reads exactly the corrected layout parses a 3-segment commit (hard and soft deletes, a doc-values update, user data) to the footer with every value equal to Lucene's reader: 4 of 4 pass, -ea and -da. The stock minimal test fails under gradle; it keeps failing after the fix by design, since the fix changes no bytes.

Add the four missing elements, type NameCounter as vLong, fix the SegID sentence and the Int64 row, and add one sentence such as "All fixed-width integers in this file are written big-endian (see CodecUtil#writeBEInt)" instead of relinking each row.

Trade-offs

Smaller diff; the per-row links still point at little-endian writers.

What the checks validated

Not run (text only).

CodecUtil's header and footer javadoc links DataOutput#writeInt / #writeLong for its big-endian Magic, Version, AlgorithmID and Checksum fields (CodecUtil.java:60, :64, :399-403).

Trade-offs

Closes the same convention gap at its source; widens the change beyond segments_N.

What the checks validated

Not run.

Drop the field list and point readers at the source.

Trade-offs

No duplicated layout left to drift (the pointer itself can still go stale), but the file-formats overview would then link a page with no format on it, against its own promise ("defines the index file formats"). Not recommended.

What the checks validated

Not run.

• Method (semi-formal). Every file grammar is data in a small grammar language: the documented grammar transcribed only from the class javadoc, the layout grammar from the writer. A generic interpreter parsed the real bytes of 64 generated default-codec index configurations (fixed and seeded random; 1,114 file checks) under both and compared every parsed value with Lucene's reader. Positive control: the layout grammar parses every file exactly to the footer. Vacuity check: 14 of 14 single-element layout mutants are killed.
• Result for this file. 64 of 64 segments_N files diverge under the documented grammar, also under a big-endian reading of it (2026-10-28).
• Lead. The hunt started from a probe of the .fnm format only; segments_N came from the grammar check.
• Then. A reviewer from another model family (OpenAI GPT-5.6) wrote and ran its own test; independent verifiers, an intended-lens reviewer and an adjudication re-ran or rebuilt the evidence.