A postings stream that ends at the very last byte of the 2 GiB indexing pool reads back with up to 3 wrong bytes (int overflow in ByteSliceReader)

During indexing, each thread buffers its postings (doc deltas, frequencies, positions, payloads) in one in-RAM byte pool of 32 KiB blocks, as chains of small "slices"; at flush ByteSliceReader walks each chain and follows the forwarding address at the end of every slice but the last. Its test for "the next slice is the last one", nextIndex + newSize >= endIndex (ByteSliceReader.java:109), is computed in int. The pool deliberately admits its 65,536th block, whose last byte is global offset 2^31 - 1, and the allocator places a slice that fits exactly at the end of a block, so a stream's final slice can end exactly at 2^31. The sum is then 2^31, which wraps negative; the reader treats the final slice as non-final and, if the stream wrote any of the slice's last 3 data bytes, returns those 1-3 bytes from an unrelated place in the pool.

Through IndexWriter with the default config and codec, one document with payloads that fills the pool this way commits normally, a 355-byte payload reads back with its last 3 bytes wrong, and CheckIndex reports the segment clean. A slightly larger input fails loudly with the ArithmeticException that #12392 added for this regime. The expression dates from LUCENE-843 (2007, first released in 2.3.0); comparing lengths (endIndex - nextIndex <= newSize) fixes it. Priority is low: it needs about 2 GB of postings in one indexing call and exact alignment.

1. The pool can be exactly 2^31 bytes. ByteBlockPool.nextBuffer advances byteOffset = Math.addExact(byteOffset, BYTE_BLOCK_SIZE) (ByteBlockPool.java:199): the 65,536th block (offset 2^31 - 32,768) is admitted, the next throws. Upstream TestByteBlockPool.testTooManyAllocs asserts this boundary.
2. A slice can end at the block's last byte. ByteSlicePool.allocKnownSizeSlice opens a new block only if pool.byteUpto > BYTE_BLOCK_SIZE - newSize (ByteSlicePool.java:117-119), so an exact fit stays; the last byte gets the marker 16 | level (:137), and the previous slice's last 4 bytes the forwarding address (:134). newSlice uses the same rule (:79).
3. The overflow. ByteSliceReader.nextSlice reads the address (ByteSliceReader.java:98) and the next size (:101), then tests if (nextIndex + newSize >= endIndex) (:109). endIndex, the stream's write position, is at most 2^31 - 1, because the slice's last byte is the marker. For the slice [2^31 - newSize, 2^31) the sum is 2^31, which wraps to -2,147,483,648: the test is false, and the reader sets limit = upto + newSize - 4 (:116), as for a slice that points onward. The assert endIndex - nextIndex > 0 (:111) sits on the other branch, so -ea does not catch it.
4. The misread. If the stream stops before the slice's last 3 data bytes, the reader stops at endIndex first and nothing goes wrong. If it wrote into any of them, the reader reaches limit (readByte :73, readBytes :129), calls nextSlice again, reads those bytes plus the marker as a little-endian address (:98) and returns the stream's last 1-3 bytes from there. (For a few byte values that address lands so close to a block's end that the remaining 2-3 bytes do not fit; then the read runs off the 32 KiB block and throws ArrayIndexOutOfBoundsException instead, as a probe on main showed.)
5. How IndexWriter gets there. At flush, FreqProxFields reads each term's doc/freq and position/payload streams through TermsHashPerField.initReader (TermsHashPerField.java:94) and this reader (FreqProxFields.java:320, :435-436). Term bytes and postings share one pool (TermsHash.java:50-55). The per-thread RAM hard limit is checked only after a whole document (DocumentsWriter.java:427-434, DocumentsWriterFlushControl.java:239), so one document can fill the pool to its last byte.
6. The init twin. init has the same pattern, startIndex + firstSize >= endIndex (:58), but TermsHashPerField.initStreamSlices always leaves room after a first slice (TermsHashPerField.java:158-162), so the indexing chain cannot produce that layout. Changing it is for consistency only.

Boundary (executed by the tie-break's own controls): 11 bytes read back correctly; 12, 13 and 14 bytes give 1, 2 and 3 wrong bytes; the same 14 bytes one block earlier read back correctly.

• Defaults. Default IndexWriterConfig (16 MB RAM buffer, 1,945 MB per-thread hard limit) and the default codec, executed end to end. The payloads come from the public TokenStream API, not a setting.
• Scale and alignment. With the 16 MB buffer the thread flushes after any larger document, so one document must bring about 2 GiB of term bytes and postings by itself. One stream's final slice must be the exact-fit slice ending at byte 2^31, with the stream's last write in that slice's last 3 data bytes, and the call must end before anything else allocates from the pool (the next allocation throws the known ArithmeticException). A planned token stream hits it deterministically; ordinary workloads essentially never. Reproducing end to end takes an 8 GiB heap and about 6 s.
• Large RAM buffers (code reading, not executed): the hard limit is checked only between documents, so with a large non-default buffer earlier documents can pre-fill the pool and the crossing document needs only the rest, about 103 MiB with the default 1,945 MB hard limit and about 1 MiB with the maximum 2,047 MB. Exact alignment is still needed.
• What goes wrong. Up to 3 wrong bytes at the end of one term's stream are committed with no error, and CheckIndex cannot tell. Executed: the last 3 bytes of a payload. Code reading only: the bytes could also be the last position or offset codes, and a hit in a doc/freq stream could change a doc delta or frequency or make the flush fail.
• Not tested, not claimed: term vectors (TermVectorsConsumerPerField.java:114-117 uses the same reader), MemoryIndex's similar test on its int pool (MemoryIndex.java:360), and FSDirectory (the wrong bytes come from the in-RAM buffer before the codec writes, so the directory should not matter).
• Bounds. The bogus address's high byte is the slice's level marker (0x11 to 0x19), so it always points 272-416 MiB into the full pool (code reading; the runs that print it agree). No hang, no memory unsafety. In the rare byte-value case above the read throws ArrayIndexOutOfBoundsException instead; through IndexWriter that would be an exception at flush (not executed).

The allocator hands out one shared zeroed 32 KiB array for the first 65,535 blocks (the memory-saving pattern of upstream TestByteBlockPool.testTooManyAllocs) and a fresh array for the last, so the test needs about 64 KiB of block memory. A filler slice of 32,768 - 5 - 14 bytes makes the stream's 5-byte and 14-byte slices tile the rest of the last block. The write loop is upstream TestByteSliceReader's.

The shared array is not what causes the failure: a full-size test with the stock DirectTrackingAllocator, all 65,536 real blocks and the real TermsHashPerField write path fails the same way, and so does the public-API end-to-end program (one document, default config and codec, 8 GiB heap); both are impact evidence, not the repro. As a regression test it is the same test: it fails on stock and passes with the fix, and needs no @Monster annotation.

# gradle, stock 645d4fe14a20, seed DEADBEEF, assertions on and off (same output)
TestByteSliceReaderFinalSliceAtPoolEnd > testFinalSliceEndingAtTwoToThe31 FAILED
    arrays first differed at element [11]; expected:<12> but was:<0>
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:8031BDBC8E36FC30]:0)
        at app//org.junit.internal.ComparisonCriteria.arrayEquals(ComparisonCriteria.java:78)
        ...
        at app//org.apache.lucene.index.TestByteSliceReaderFinalSliceAtPoolEnd.testFinalSliceEndingAtTwoToThe31(TestByteSliceReaderFinalSliceAtPoolEnd.java:59)
        Caused by:
        java.lang.AssertionError: expected:<12> but was:<0>

# with the fix, assertions on and off
:lucene:core:test (SUCCESS): 1 test

# impact evidence: public-API end-to-end program, one document, default config and codec
indexed; files: [_0.cfe, _0.cfs, _0.si, segments_1]
term b: freq=2, payload 1 (355 bytes) differs [352]: indexed 0x41, read 0x11 [353]: indexed 0x42, read 0x11 [354]: indexed 0x43, read 0x11;
CheckIndex clean=true
# control (that payload 3 bytes shorter): term b: freq=2, positions and payloads OK; CheckIndex clean=true

• The reader's contract. "We read the bytes in each slice until we hit the end of that slice at which point we read the forwarding address of the next slice and then jump to it" (ByteSliceReader.java:25-29).
• The public contract. PostingsEnum.getPayload: "Returns the payload at this position" (PostingsEnum.java:96-100). IndexWriter.addDocument documents no per-document size limit other than MAX_TERM_LENGTH (IndexWriter.java:1480-1481).
• The state is legal. The pool admits the 65,536th block and only the next throws (ByteBlockPool.java:199, asserted by TestByteBlockPool.testTooManyAllocs); CHANGES.txt: "Each DWPT can address up to 2048 MB memory" (CHANGES.txt:15722-15729). Every offset fits in an int; only the intermediate sum does not.
• Upstream's answer for this regime is a loud failure. "GITHUB#9660: Throw an ArithmeticException when the offset overflows in a ByteBlockPool", under Bug Fixes (CHANGES.txt:3445, #12392); IndexingChain turns it into an aborting exception (IndexingChain.java:2098-2100). This exact fit slips past that guard.
• Nothing else is exhausted. With the fix, the same full-pool document indexes and reads back exactly (executed). No javadoc, test, CHANGES entry or discussion accepts the wrapped comparison.

The strongest case for "unsupported": setRAMPerThreadHardLimitMB calls the hard limit a safety limit against "address space exhaustion due to its internal 32 bit signed integer based memory addressing. The given value must be less that 2GB (2048MB)" (IndexWriterConfig.java:374-378); DocumentsWriterFlushControl.java:240-241 says "we can not address more than 2048 MB per DWPT"; on PR #15330 rmuir wrote "But this is not supported: the limits on IndexWriter are 2GB"; on #10790 mikemccand called the hard limit "best effort" and said a gigantic document "can lead to exceptions like this"; and the classes are package-private.

Why it fails: the failing state is inside the documented 2,048 MB, Lucene itself lets one document reach it, and every statement above accepts an exception there, never silent wrong postings that CheckIndex accepts. It argues for low priority, not for "not a bug". The works-as-designed reviewer from another model family (OpenAI GPT-5.6) rated the argument moderate and still called the bug real.

The study's two-line fix was run through gradle in a scratch worktree of 645d4fe14a20 (OpenJDK 25.0.3, seed DEADBEEF).

At ByteSliceReader.java:109 and :58. Because 0 <= nextIndex < endIndex <= 2^31 - 1 (and startIndex <= endIndex), the differences cannot overflow; the test is otherwise the same. No allocator or format change. The :58 line cannot be reached through the indexing chain and is changed for consistency.

-    if (startIndex + firstSize >= endIndex) {
+    if (endIndex - startIndex <= firstSize) { // no int overflow when the slice ends at 2^31
       // There is only this one slice to read
@@
-    if (nextIndex + newSize >= endIndex) {
+    if (endIndex - nextIndex <= newSize) { // no int overflow when the slice ends at 2^31
       // We are advancing to the final slice

Trade-offs

• Two comparisons on the flush read path; no measurable cost expected, no benchmark run.
• MemoryIndex.SliceReader has a similar test on its int pool (MemoryIndex.java:360); it was not tested and is not part of this fix.

What the checks validated

• Stock: the repro fails with assertions on and off.
• Fix: the repro passes with assertions on and off.
• The closest core suites (TestByteSliceReader, TestByteSlicePool, TestTermsHashPerField, TestByteBlockPool, TestPayloads, TestPayloadsOnVectors, TestPostingsOffsets, TestTermVectorsWriter, TestDocumentWriter, TestFlushByRamOrCountsPolicy, TestIndexWriterExceptions): 11 classes, 109 tests, 0 failures, 3 skipped.
• The whole core suite with the repro in the tree: 749 classes, 8,898 tests, 0 failures, 322 skipped.
• :lucene:core:check -x test with the repro in the tree passes (google-java-format, ecjLint, forbiddenApis).
• Earlier, shadowed onto the stock jars: the minimal test, the tie-break's boundary controls (5/5) and, on the study commit, the study's package-level repros and upstream TestByteSliceReader, TestByteSlicePool and TestTermsHashPerField pass; the same full-pool end-to-end document reads back exactly with the fixed reader.

(long) nextIndex + newSize >= endIndex at :109 and (long) startIndex + firstSize >= endIndex at :58: same effect, keeps the original shape.

What the checks validated

By the study's independent reproducer on its commit: its JUnits, the full-size test with the stock allocator and the end-to-end program pass. Not run through gradle here.

Make the allocator open no exact-fit slice at the end of block 65,535.

Trade-offs

Changes the allocator for one edge case and leaves an overflow-prone test in the reader. Not written or run.

A check in doAfterDocument.

Trade-offs

Runs between documents, so one document can still fill the pool; does not fix this.

Method: formal (an SMT model produced the signal), a hunt unit on Lucene's byte and int pools.

• A Z3 bit-vector model of ByteSlicePool and ByteSliceReader (query group Q4): Q4c is satisfiable at every slice level with block 65,535 and a slice ending at 2^31; Q4c' (a wrap anywhere else) is unsatisfiable; Q4d / Q4d' show the reader follows a bogus address if and only if one of the slice's last 3 data bytes is used; Q4e' shows TermsHashPerField.initStreamSlices cannot produce the init twin.
• A bounded-exhaustive round-trip harness below the pool end (3,564,378 write schedules, 6,300,936 read-backs in 5 read modes) found no other defect.
• The witness was executed as a JUnit through the real TermsHashPerField write path; an independent verifier then wrote a full-size test with the stock allocator and a public-API end-to-end program, using a planner that replays the indexing chain's pool writes to compute the token schedule.