The default block tree never sets Stats.indexNumBytes, so CheckIndex prints "index trie: 0 bytes" for every field

The default terms dictionary since 10.3.0, the Lucene103 block tree, never assigns Stats.indexNumBytes, documented as "Byte size of the index". SegmentTermsEnum.computeBlockStats() creates the Stats object and walks the whole terms dictionary to fill the other counters, but never sets this one, so it keeps Java's default 0. Terms.getStats() therefore reports 0, and CheckIndex -verbose at level 2 or 3 prints index trie: / 0 bytes for every field that has terms, for example for a field of 10,000 ids whose trie takes 1,197 bytes of a 1,269-byte .tip file. FieldReader already holds the trie's byte range (indexStart, indexEnd). The line that set the field was dropped when #14333 copied the Lucene90 block tree to create the Lucene103 one. Nothing else reads the value, so this is a small diagnostic defect; and since 8.6.0 the older block tree printed a size-independent constant there, so it is not a regression of a working number.

// SegmentTermsEnum.java:87-91
/** Runs next() through the entire terms dict, computing aggregate statistics. */
public Stats computeBlockStats() throws IOException {
  Stats stats = new Stats(fr.parent.segment, fr.fieldInfo.name);   // indexNumBytes stays 0
  currentFrame = staticFrame;
  ...                                                              // fills the term and block counters

• Contract: Stats.java:34-35, /** Byte size of the index. */ public long indexNumBytes;, printed by Stats.toString under index trie: as N bytes (Stats.java:184-185). Stats is @lucene.internal (Stats.java:31).
• Nobody writes it: in lucene/core, lucene/backward-codecs, lucene/codecs and lucene/test-framework sources the only write is in the old Lucene90 copy, backward_codecs/lucene90/blocktree/SegmentTermsEnum.java:124-127: if (fr.index != null) { stats.indexNumBytes = fr.index.ramBytesUsed(); }.
• The size is known: FieldReader reads indexStart, rootFP and indexEnd from the terms metadata (FieldReader.java:80-82) and opens the trie as a slice of indexEnd - indexStart bytes of .tip (:86-87). TrieBuilder.save writes the nodes and 8 padding bytes ("additional 8 bytes for over-reading") to .tip, and records indexStart, rootFP and indexEnd in the terms metadata file (TrieBuilder.java:263-268), so one term gives a 10-byte range.
• Why the line went missing (a reading of the mechanics, not a stated intent): the Lucene103 FieldReader has no index object to call ramBytesUsed() on; it builds a TrieReader on demand.
• Where it shows: FieldReader.getStats() ("For debugging -- used by CheckIndex too", FieldReader.java:110-114). CheckIndex stores each field's Stats (CheckIndex.java:2254-2259) and prints them only with verbose output (:2488-2493). It collects them in testPostings, which runs only at level >= MIN_LEVEL_FOR_INTEGRITY_CHECKS = 2 (:1114, :1125, :4754); the command-line default is level 1 (:4748).

CheckIndex at level 2 with verbose output prints index trie: / 0 bytes for both and reports the index clean; with the fix it prints 10 bytes and 1197 bytes.

• Default: the default codec, an ordinary IndexWriter; the smallest case is one document with one term.
• Effect: one wrong number in debugging output. No wrong results, no index damage, no exception. CheckIndex's checks next to it recompute sumDocFreq and sumTotalTermFreq themselves (CheckIndex.java:2261-2281) and never read the Stats fields. Nor does the 0 hide a resident heap structure: FieldReader keeps only the trie's offsets (FieldReader.java:44-46) and builds a TrieReader over an IndexInput slice of .tip on demand (:86-87); a terms enum allocates only small temporary Node objects while it walks (TrieReader.java:78-82, SegmentTermsEnum.java:58, :177-186). Where the .tip bytes live is up to the Directory (the repro's ByteBuffersDirectory keeps them on the heap).
• Scale: independent of the index: 0 for a 2-term field and for a 200,000-term field alike.
• People read this line: 2015-16 JIRA issues (now GitHub #8476 and #7254) paste CheckIndex output with real index FST: N bytes values.
• Likelihood: every such diagnostic run on a 10.3+ index shows the 0, but these are occasional debugging actions, nothing acts on the number, and nobody has reported it in the roughly 14 months since 10.3.0.

• Smallest input. One document, one term: the Lucene103 writer always saves the field's trie to .tip (10 bytes here), and the test passes with the fix, so one term already shows the bug.
• API. Public only: IndexWriter, DirectoryReader, Terms.getStats() cast to the public Stats class. The package org.apache.lucene.codecs.lucene103.blocktree is for locality; no package-private API is used.
• Releases. The same test fails on the 10.6.0 release jars with assertions on and off; the hunt's earlier two-method test (10,000 ids, plus a CheckIndex check) failed on every release from 10.3.0 to 10.5.2.
• As the regression test: this same method. The closest upstream test, testFinalBlock in TestLucene104PostingsFormat, already reads the same Stats. The repro's one comment line is 103 characters, so checkGoogleJavaFormat asks to wrap it (./gradlew tidy does).

cp $REPRO/TestStatsIndexNumBytes.java $LUCENE/lucene/core/src/test/org/apache/lucene/codecs/lucene103/blocktree/
cd $LUCENE
./gradlew :lucene:core:test --tests org.apache.lucene.codecs.lucene103.blocktree.TestStatsIndexNumBytes -Ptests.seed=DEADBEEF
# assertions off: add -Ptests.asserts=false (same failure)

# gradle, main 645d4fe14a20, seed DEADBEEF, assertions on
TestStatsIndexNumBytes > testIndexNumBytesIsSet FAILED
    java.lang.AssertionError: expected indexNumBytes > 0 but was 0
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:15DBA2BD6AD76884]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.assertTrue(Assert.java:42)
        at org.apache.lucene.codecs.lucene103.blocktree.TestStatsIndexNumBytes.testIndexNumBytesIsSet(TestStatsIndexNumBytes.java:43)
1 test completed, 1 failed
BUILD FAILED
# gradle, assertions off (-Ptests.asserts=false): the same AssertionError
# 10.6.0 release jars, assertions on and off: the same AssertionError
# with the recommended fix: BUILD SUCCESSFUL, 1 test, 0 failures (assertions on and off)

• Contract: "Byte size of the index" (Stats.java:34) and the label index trie: N bytes (:184-185). The neighbouring counters are on-disk sizes: totalBlockSuffixBytes "Total number of bytes used to store term suffixes" (:73-74), totalBlockStatsBytes (:86-89), totalBlockOtherBytes (:92-95).
• History of the field: the first block tree (LUCENE-3030, 4.0.0-alpha) set it with stats.indexNumBytes = index.sizeInBytes(), and the block trees up to 10.2 kept an assignment (a probe reads non-zero values on 7.7.3 to 10.2.2). #14333 created the Lucene103 copy without it.
• What it means: in a 2013 java-user thread ("Understanding FST Prefix & CheckIndex output") a user asked whether "index FST bytes" is the field's part of the .tip file, and Michael McCandless answered "Yes."
• Precedent: in #13330 (2024) an earlier revision of the PR removed the same assignment from the then-core Lucene90 block tree; benwtrent asked in review whether that was wanted, and the PR's author, jpountz, put it back before merge, calling the removal "a leftover from a previous change". When LUCENE-6617 (2015) made the FST node and arc counts unavailable, the matching Stats fields were removed rather than left at 0.
• No sign of intent: no CHANGES or MIGRATE entry, test or code comment says 0 is intended, and no description, review or comment of #14333 or #14563 mentions Stats, indexNumBytes or getStats. The two upstream tests that read this Stats class, testFinalBlock in TestLucene104PostingsFormat and its copy in backward-codecs TestLucene103PostingsFormat, check block counts only.

The strongest case for "intended": the Lucene90 value was FST.ramBytesUsed(), a heap figure, and the Lucene103 field reader keeps no resident trie on the heap (only its offsets; a terms enum allocates temporary nodes), so 0 could mean "no resident heap"; Stats is @lucene.internal and getStats() a debugging API; and the old number had been a meaningless constant for years. The cross-model reviews (OpenAI GPT-5.6) rated this intended-behaviour argument "moderate", and both still judged the bug real with high confidence.

Why it fails: the javadoc says "Byte size of the index", not heap; the 4.0 original measured the index size; the sibling counters are on-disk bytes; and a field that nobody assigns is not a measurement under any reading. The internal/debug status lowers the severity, not the correctness. The constant-before-10.3 point is real, so this is not a regression of a working number.

One fix was built and validated; the alternatives were not run.

One reasonable reading of "Byte size of the index": the field's part of the .tip file, exactly the slice FieldReader opens (FieldReader.java:86-87), from metadata the reader already holds.

// SegmentTermsEnum.java:88-90
   public Stats computeBlockStats() throws IOException {
     Stats stats = new Stats(fr.parent.segment, fr.fieldInfo.name);
+    stats.indexNumBytes = fr.indexEnd - fr.indexStart;

Trade-offs

• Includes the 8 padding bytes TrieBuilder.save writes (TrieBuilder.java:266), so one term reports 10 bytes.
• The added line is O(1), though getStats() itself still walks the whole terms dictionary; debugging output only, no hot path changes.
• It turns the old heap-accounting figure into an on-disk extent. It fits the javadoc and the 2013 java-user answer, but maintainers may prefer B, a relabelled heading, or C.
• Lucene103-format segments only: Lucene90-format segments read through backward-codecs keep their old ramBytesUsed() figure (see the aside).

What the checks validated

• Gradle, scratch worktree of 645d4fe14a20, OpenJDK 25.0.3, seed DEADBEEF: the minimal test fails before the change and passes after it, with assertions on and off.
• :lucene:core:test over org.apache.lucene.codecs.lucene103.*, org.apache.lucene.codecs.lucene104.* (incl. TestLucene104PostingsFormat), TestCheckIndex, TestTermsEnum* and TestTerms: 15 classes (including the new test), 212 tests, 8 skipped, 0 failures.
• :lucene:core:checkGoogleJavaFormat with only the fix applied: clean.
• A probe with the fix prints index trie: 1197 bytes for 10,000 ids and 10 bytes for one term; unfixed main prints 0 for both.
• Earlier and independently: the hunt's fix check (2 of 2 pass), TestLucene104PostingsFormat and TestCheckIndex (41 tests, 0 failures), and the cross-model reviewer's own test pass with it.

stats.indexNumBytes = fr.indexEnd - fr.indexStart - Long.BYTES; reports only the trie nodes (2 bytes for one term). Not run.

Trade-offs

More exact, but it ties Stats to a layout detail of TrieBuilder.save that could change.

As LUCENE-6617 did for the FST node and arc counts. Not run.

Trade-offs

Stats is @lucene.internal, so the API cost is small, but CheckIndex loses the one per-field terms-index size it prints.

The backward-codecs Lucene90 computeBlockStats could use fr.index.numBytes() (FST.java:506-508, the FST's serialized size) instead of ramBytesUsed(), which for an off-heap FST is a constant (FST.java:496-499, OffHeapFSTStore.java:46-47). Not run.

Trade-offs

Changes the output for 9.x-format segments; independent of fix A.

• First signal: code reading in the hunt unit on the Lucene103 block tree. A grep for writes to indexNumBytes in lucene/core found only the declaration and the read in Stats.toString. No model produced or witnessed it, so the method is non-formal.
• Repro: the finder's JUnit (10,000 StringField ids in one segment, getStats() and CheckIndex verbose at level 2) failed 2 of 2 on main with assertions on and off, on 10.5.2 and on 10.3.0, and passed with the one-line fix.
• Minimal repro (2026-10-29): cut to one document with one term and one test method; fails on main 645d4fe14a20 with assertions on and off.