MultiPostingsEnum.toString() throws NullPointerException when the term is missing from a segment that has the field

Reading one term's postings across a multi-segment reader (MultiTerms.getTermPostingsEnum, or MultiTerms.iterator() and postings()) returns a MultiPostingsEnum; a single-segment reader gets the segment's own enum instead (MultiTerms.java:71-73). Unless it is handed a reusable enum, MultiTermsEnum.postings creates it with one slot per segment that has the field (MultiTermsEnum.java:324, :327), and the constructor fills every slot with an empty EnumWithSlice whose slice is null (MultiPostingsEnum.java:44-51 at 5adff78bb168). reset then fills only the first numSubs slots, one per segment that holds the term (:59-69). nextDoc, advance, cost and the only other reader of getSubs() stop at numSubs, but toString() prints the whole array (:193), and EnumWithSlice.toString() calls slice.toString() (:178). So for any term missing from at least one segment that has the field, toString() on a newly created enum (getTermPostingsEnum always creates one) throws NullPointerException, with the default configuration and with assertions on and off, while the postings themselves are correct. A segment without the field gets no slot and does not trigger it. The null slots came with LUCENE-5798 (2014-07-01, first released in 4.10.0), a reuse optimization; before it every slot carried a slice. Every executed release from 4.10.0 to 10.6.0 and main fail; 4.9.0 and 4.9.1 pass. No production code in the Lucene source tree calls this toString(), so the effect is limited to logging, error messages and debuggers. A one-line fix prints only the first numSubs slots. Priority Low.

1. One slot per segment. Unless it is handed a reusable enum made by the same MultiTermsEnum, MultiTermsEnum.postings creates new MultiPostingsEnum(this, subs.length) (MultiTermsEnum.java:316-328: :324, :327); subs has one entry per segment that has the field (MultiTerms.java:78-85). The constructor fills every slot with an empty EnumWithSlice, slice and postingsEnum both null (MultiPostingsEnum.java:44-51).
2. Only matching segments are filled. postings packs the segments positioned on the term into subDocs (MultiTermsEnum.java:334-347) and calls reset(subDocs, upto) (:349); reset copies only the first numSubs slots (MultiPostingsEnum.java:59-69).
3. Everything else honours numSubs. getNumSubs(): "How many sub-readers we are merging. @see #getSubs" (:71-78); advance and nextDoc stop at numSubs - 1 (:113, :127); cost loops to numSubs (:185); MappingMultiPostingsEnum.reset, the only other reader of getSubs(), stops at getNumSubs() (MappingMultiPostingsEnum.java:67-72).
4. toString() does not (the fix site). "MultiDocsAndPositionsEnum(" + Arrays.toString(getSubs()) + ")" (:193) prints every slot, and EnumWithSlice.toString() is slice.toString() + ":" + postingsEnum (:178), which throws on a null slice.

In the repro, the diagram probe shows getNumSubs() = 1 and getSubs().length = 2; slot 0 holds segment 0's slice and postings, slot 1 has slice = null; cost() is 1 and nextDoc() gives 0, then NO_MORE_DOCS. For the term b (only in segment 1) slot 0 holds segment 1 and slot 1 is again empty: the slots are packed, so slot numbers are not segment numbers. With a in both segments, getNumSubs() = 2 and toString() works. A fact-check probe with a second segment that lacks the field f altogether found 1 slot, getNumSubs() = 1, and a toString() that prints: a segment without the field gets no slot.

A side effect of the same line: reset never clears the tail, so a reused enum prints a slot left over from an earlier term instead of throwing (seen once in the independent reproducer's control). The same fix covers it; it is not a separate claim.

• Who. Application or test code that calls toString() on what MultiTerms.getTermPostingsEnum or MultiTerms.iterator().postings() returns, directly, by string concatenation or through a debugger. No production path in the Lucene source tree calls it; downstream code was not checked.
• Default vs opt-in. No special configuration: the diagram probe ran outside the test framework with the default IndexWriterConfig and codec, and the JUnit runs failed with whatever codec the framework drew from each seed. A term missing from a segment that has the field is the normal state of a multi-segment index; a single-segment reader returns the segment's own enum and is not affected (MultiTerms.java:71-73). MultiPostingsEnum and MultiTerms are @lucene.experimental.
• Effect. One unchecked NullPointerException from toString(); no wrong results, no index change, no hang.
• Scale. Two one-document segments.
• Likelihood: rare. It needs an explicit toString() of this enum, and no matching public report was found in the twelve years since 4.10.0.

TestMultiPostingsEnumToString (one method, extends LuceneTestCase, package org.apache.lucene.index of the lucene/core tests). It uses a ByteBuffersDirectory and new IndexWriterConfig() with NoMergePolicy, so the method itself draws no random values; the framework still picks the default codec, similarity and locale from the seed, and it failed on every recorded seed. About 1 s.

As a regression test the same method fits in TestMultiFields; checkGoogleJavaFormat flags only its 108-character comment line.

cp $REPRO/TestMultiPostingsEnumToString.java $LUCENE/lucene/core/src/test/org/apache/lucene/index/
cd $LUCENE && ./gradlew :lucene:core:test --tests org.apache.lucene.index.TestMultiPostingsEnumToString -Ptests.seed=DEADBEEF
# and with -Ptests.asserts=false: the same failure

TestMultiPostingsEnumToString > testTermMissingFromOneSegment FAILED
    java.lang.NullPointerException: Cannot invoke "org.apache.lucene.index.ReaderSlice.toString()" because "this.slice" is null
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:1914E18AA0417813]:0)
        at org.apache.lucene.index.MultiPostingsEnum$EnumWithSlice.toString(MultiPostingsEnum.java:178)
        at java.base/java.lang.String.valueOf(String.java:4530)
        at java.base/java.util.Arrays.toString(Arrays.java:4963)
        at org.apache.lucene.index.MultiPostingsEnum.toString(MultiPostingsEnum.java:193)
        at org.apache.lucene.index.TestMultiPostingsEnumToString.testTermMissingFromOneSegment(TestMultiPostingsEnumToString.java:43)
1 test completed, 1 failed

# with -Ptests.asserts=false: the same exception and frames

• Contract. getNumSubs(): "How many sub-readers we are merging. @see #getSubs" (MultiPostingsEnum.java:71-75); getSubs(): "Returns sub-readers we are merging." (:80), yet it returns the whole slot array. toString() has no javadoc. MultiTerms.getTermPostingsEnum is a public static helper with no precondition that the term be in every segment (MultiTerms.java:106-124).
• History. Today's MultiPostingsEnum is the old MultiDocsAndPositionsEnum, renamed by LUCENE-4524 (ad18a0ea95e on branch_5x, first released in 5.1.0; c13216934c5 on trunk), which also removed its sibling MultiDocsEnum. Both of its toString() methods (the enum's and EnumWithSlice's) came with 24092aab4cd ("add toString for debugging", Robert Muir, 2012-07-21); MultiDocsEnum had had the same pair since 21b7c5bbee1 (LUCENE-3584, 2011-11-30). Until 4.9.1, reset() built new slots by copying all subs.length entries of MultiTermsEnum's array, whose slices are always set (MultiTermsEnum.java:69-74). LUCENE-5798 ("Optimize MultiDocsEnum reuse", 489b8e82a8c on branch_4x, 04ef241474d on trunk, Robert Muir, 2014-07-01; #6860; listed under 4.10.0 Optimizations, lucene/CHANGES.txt:11974) pre-allocated empty slots and copied only numSubs of them. Its discussion is about garbage, never toString().
• Tests. No test under lucene/ names MultiPostingsEnum or EnumWithSlice, so nothing checks this toString().
• Precedent. "LUCENE-8130: Fix NullPointerException from TermStates.toString()" is listed under 7.3.0 Bug Fixes (lucene/CHANGES.txt:7696, #9178); #3600, #2357 and #2323 fixed other toString() exceptions as bugs.

The strongest case for "intended": toString() is a debugging aid on an @lucene.experimental class, and it simply dumps the raw slot array.

Why it fails: @lucene.experimental is about API stability, not about a valid call throwing; a dump that throws on the most ordinary state is not a faithful dump; the slots past numSubs belong to no segment of the current term; and the method did not throw before 4.10.0. That getSubs() hands out the whole array does not make the tail meaningful: its javadoc pairs it with getNumSubs(), and its only in-tree caller honours that count. The intended-lens cross-model review rated the "intentionally correct" case weak, and the devil's-advocate verifier rejected it. A maintainer may still call it too minor to fix, a fair outcome for a Low bug.

Fix A was built and checked through gradle on 5adff78bb168; B was not run.

In MultiPostingsEnum.toString() (:193), print a copy of the first numSubs slots, as every other member reads them. The label MultiDocsAndPositionsEnum( is the class's name from before LUCENE-4524 (5.1.0); renaming it to MultiPostingsEnum( in the same line is optional and was not run.

    return "MultiDocsAndPositionsEnum(" + Arrays.toString(Arrays.copyOf(subs, numSubs)) + ")";

Trade-offs

Copies the first numSubs slot references (one per segment that holds the term) per toString() call, which only diagnostics pay for. It also stops a reused enum from printing a slot left over from an earlier term. On the repro's state the fix's expression, evaluated by the diagram probe through getSubs() and getNumSubs(), gives one entry, segment 0's slot; the gradle fix runs show that toString() no longer throws, not the printed string.

What the checks validated

• Gradle in a scratch worktree at 5adff78bb168, seed DEADBEEF: the repro fails without the fix (assertions on and off) and passes with it (assertions off).
• With the fix and assertions on: the repro (1), TestMultiFields (3), TestLongPostings (2), TestPayloads (7) and TestPostingsOffsets (12): 25 tests, 0 failures. checkGoogleJavaFormat fails (exit 1) on the repro as written, only for its 108-character comment line; the fix line passes. The full core suite was not run, and only OpenJDK 25.0.3 on Linux aarch64 was used.
• Earlier: the hunt's runs of the same diff pass the repro with assertions on; the cross-model reviewers' runs pass their own tests and the 3 TestMultiFields tests.

slice == null ? "<unused>" : slice.toString() + ":" + postingsEnum at :178.

Trade-offs

No exception, but the output still lists slots that belong to no segment of the term, and stale slots after reuse. Not recommended.

What the checks validated

Not run.

Method: semi-formal (a proof-obligation audit of a class's contract, witnessed by an executable repro).

• The finder checked each member of MultiPostingsEnum against the slot invariant (slots below numSubs hold the segments of the current term; slots past it belong to no segment of the term: empty on a new enum, stale after reuse) and found that only toString() reads past numSubs.
• The finder then wrote a public-API JUnit test and the one-line fix that makes it pass.
• An independent reproducer, two executed cross-model reviews, a devil's-advocate verifier and an adjudication reproduced it with their own tests ("Prior reports, history and checks").