A commit() that fails after its pending segments file is synced makes rollback() delete the last commit's files

The finding in 30 seconds

commit() protects the new commit's files with one extra reference each, writes pending_segments_N, fsyncs, then logs "done all syncs". A failed fsync is undone; a throw from that log line is not. The commit's cleanup releases the reference, but the half-finished commit stays pending, so rollback() releases it again.

How a commit protects its files, in one minute

IndexFileDeleter counts references to every index file: one for each commit point that lists it, one for the writer's current segments, and extra ones a running commit takes. A file whose count reaches 0 is deleted.
commit() first prepares: it takes one extra reference on every file of the new commit, writes pending_segments_N (not yet visible to readers) and fsyncs the files. From then on the extra reference belongs to the pending commit.
Then it finishes: it renames pending_segments_N to segments_N and releases the extra reference. rollback() instead deletes pending_segments_N and releases it. Either way it is released exactly once.

A flow diagram on a light card: the top row shows three holders (the last commit segments_1, the writer's segment list, a commit in progress) that each hold one reference on a file, counted by IndexFileDeleter, and a count of 0 deletes the file; the middle row shows a commit: 1. commit() takes one reference, 2. it writes and syncs pending_segments_2 and records the pending commit, 3. a log line throws, ringed in red with a cross, where the failure path lets go of the reference but the pending commit stays, and a red dashed arrow marked commit() throws leads to 4. rollback(), which finds the pending commit and lets go of its reference a second time; a red box below says the count is one too low, so the last commit's files reach 0 and are deleted although segments_1 still lists them, and two panels contrast the expected result (rollback() keeps segments_1 and its files, a reader sees 1 document) with the actual one (only segments_1 is left and opening the index fails with CorruptSegmentInfoException). — A commit's file reference is let go twice when a log line throws after the pending segments file is synced, so rollback() deletes the last commit's files. Generated with AI, checked against the code.

Step-by-step picture of the minimal repro: commit 1 writes segments_1 with segment _0; the second commit takes an extra reference on _0 and _1, writes pending_segments_2 and fsyncs; the InfoStream throws on "done all syncs"; the commit's cleanup and then rollback() each release the extra reference, so rollback() deletes _0.cfe, _0.cfs and _0.si and only segments_1 is left; DirectoryReader.open fails with NoSuchFileException _0.si. Expected: the last commit opens with 1 document; the fix undoes the pending commit at the failure. — The minimal repro step by step: the calls on one side, the index files and their reference counts on the other, then expected versus actual and the fix. Select the still to open it at full size. Generated with AI, checked against the code.

The finding, step by step

Give IndexWriterConfig an InfoStream that throws one RuntimeException, on the message starting "done all syncs". Add 1 empty document and commit(): the index is segments_1 plus segment _0 (_0.cfe, _0.cfs, _0.si).
Add 1 more document and commit() again. Lucene writes segment _1 and pending_segments_2, syncs the 6 segment files (Directory.sync) and logs "done all syncs"; the InfoStream throws. commit() fails, isOpen() is true, getTragicException() is null.
You call rollback() and expect its promise: the index as it was at the last commit, segments_1 with 1 document.
Instead rollback() deletes _1.* and then _0.cfe, _0.si and _0.cfs; only segments_1 is left, and DirectoryReader.open fails with NoSuchFileException: _0.si. close() with commitOnClose=false does the same.
Why: the commit took one extra reference on each of its files, and after the failure it is released twice, by the commit's cleanup and again by rollback(), so _0's files lose the reference segments_1 holds. With assertions on, rollback() throws RefCount is 0 pre-decrement instead.

Who is affected

Applications that install their own InfoStream (off by default) whose message() can throw a RuntimeException, and that recover from the failed commit with rollback() or close() with commitOnClose=false. One committed document is enough, in memory or on disk. The last commit's files are deleted, so its data is gone unless there is a backup.

Mechanism, terms and numbers925 words

commit() takes one extra reference on every file of the new commit (IndexWriter.java:3857), then startCommit writes pending_segments_N and sets pendingCommitSet = true; pendingCommit = toSync (:5973, :5983-5984): from then on the pending commit owns that reference. After the fsync, still inside the try, it logs "done all syncs" (:6000-6002). If the InfoStream throws there, startCommit's catch releases nothing because pendingCommitSet is true (:6007) and leaves pendingCommit set, prepareCommitInternal's catch releases filesToCommit anyway (:3920-3929), and since the exception is not an Error the writer stays open with no tragic exception. rollback() then releases pendingCommit's files a second time (:2642-2645), so files shared with the last commit lose that commit's reference and the end of rollback() deletes them (:2670-2672): with assertions off the last good commit can no longer be opened; with assertions on rollback() fails on the deleter's RefCount is 0 pre-decrement assertion. Introduced in 4.8.0 by LUCENE-5543; affects every release through 10.6.0 and main. Only an application InfoStream that throws a RuntimeException on that one line reaches it (off by default). Fix: undo the pending commit in startCommit's catch, as a failed fsync already does (7 lines, validated).

The code

// IndexWriter.prepareCommitInternal                                   IndexWriter.java:3773
deleter.incRef(toCommit.files(false));        // :3857  one extra reference on _0.* and _1.*
filesToCommit = toCommit.files(false);        // :3909
try {
  startCommit(toCommit);                      // :3914
  ...
} catch (Throwable t) {                       // :3920
  if (filesToCommit != null) {                // :3922  still set: startCommit kept it
    deleter.decRef(filesToCommit);            // :3924  RELEASE 1
    ...  filesToCommit = null;                // :3928
  }
  throw t;
}

// IndexWriter.startCommit                                              IndexWriter.java:5911
toSync.prepareCommit(directory);              // :5973  writes pending_segments_2
pendingCommitSet = true;                      // :5983
pendingCommit = toSync;                       // :5984  the pending commit now owns the reference
try {
  directory.sync(toSync.files(false));        // :5991-5992
} catch (Throwable t) {                       // :5993  a failed fsync IS undone:
  pendingCommitSet = false; pendingCommit = null; toSync.rollbackCommit(directory); throw t;
}
infoStream.message("IW", "done all syncs: " + filesToSync);   // :6001  the InfoStream throws here
testPoint("midStartCommitSuccess");           // :6004  test-only; also only calls the InfoStream
} catch (Throwable t) {                       // :6005
  if (!pendingCommitSet) { ... decRef(filesToCommit); filesToCommit = null; }  // :6007-6018  skipped
  throw t;                                    // :6021  pendingCommit stays set
}

// IndexWriter.rollbackInternalNoCommit                                 IndexWriter.java:2601
if (pendingCommit != null) {                  // :2642
  pendingCommit.rollbackCommit(directory);    // :2643  deletes pending_segments_2
  deleter.decRef(pendingCommit);              // :2645  RELEASE 2 of the same reference
}
deleter.checkpoint(segmentInfos, false);      // :2670  back to [_0]
deleter.refresh();                            // :2671
deleter.close();                              // :2672  the writer's own references go: _0.* reach 0

Step by step

After the first commit each _0.* file has two references: the commit point segments_1 and the writer's current segments (IndexFileDeleter.checkpoint, IndexFileDeleter.java:544-591). Flushing _1 adds the writer's reference on _1.*; prepareCommitInternal adds one more on every file of the new commit, _0.* and _1.* (IndexWriter.java:3857). (These are the repro's counts: with merge-on-commit enabled, preparePointInTimeMerge also moves references, :3993, :4007; the repro does not use it.)
startCommit writes pending_segments_2 and sets pendingCommitSet and pendingCommit (:5973-5984). From here the pending commit holds the extra reference until finishCommit releases it through filesToCommit (:4287-4288) or rollback() through pendingCommit (:2642-2645).
A failing directory.sync resets both fields and deletes the pending file (:5993-5997). The statements after it, the "done all syncs" message (:6000-6002) and testPoint("midStartCommitSuccess") (:6004, a no-op unless test points are enabled, and then it also only calls the InfoStream, :6132-6137), have no such undo. In production only the InfoStream can throw there.
The catch releases filesToCommit only if (!pendingCommitSet) (:6007), so it rethrows with pendingCommit and filesToCommit both still set. Only an Error is tragic (:6031-6033): after a RuntimeException the writer stays open and getTragicException() is null.
prepareCommitInternal's catch sees filesToCommit != null and releases it (:3920-3929): release 1.
rollback() -> rollbackInternalNoCommit deletes pending_segments_2 and releases pendingCommit's files (:2642-2645; IndexFileDeleter.decRef(SegmentInfos) is decRef(files(false)), IndexFileDeleter.java:623-626): release 2 of the same reference. _1.* reach 0 and are deleted (harmless, uncommitted); _0.* are left one reference short.
The checkpoint back to [_0], refresh() and close() (:2670-2672) drop the writer's own references: _0.* reach 0 and are deleted although segments_1 lists them. With assertions on, the checkpoint's release of _1.cfs trips assert count > 0 (FileDeleter.java:276-282) first, so rollback() throws and the old commit survives: not a successful rollback, but a failing one that stops before it deletes _0.*.

A throw before pendingCommitSet = true (for example on the "startCommit: wrote pending segments file" line, :5974-5981) releases the references exactly once: the catch releases filesToCommit and nulls it, and prepareCommitInternal's guard skips its release. That path does not call rollbackCommit, so the pending_segments_N file prepareCommit already wrote is not deleted there (:6005-6018); what happens to it later was not checked and is not part of this bug. An Error at the "done all syncs" line is tragic and keeps the last commit intact.

The numbers (main, assertions off)

Files after commit 1: _0.cfe, _0.cfs, _0.si, segments_1. After the failed commit 2: _0.*, _1.*, pending_segments_2, segments_1; isOpen=true, tragic=null. During rollback() the deleter logs "now delete 3 files: [_1.cfs, _1.cfe, _1.si]" (release 2), then "rollback: infos=_0(12.0.0):c1", then "now delete 3 files: [_0.cfe, _0.si, _0.cfs]". Files after rollback(): segments_1 only. With the fix the failed commit removes pending_segments_2, and after rollback() the files are _0.cfe, _0.cfs, _0.si, segments_1 and the index opens with 1 document.

Exposure and likelihood390 words

Who is hit. Applications that install their own InfoStream through IndexWriterConfig.setInfoStream(InfoStream) or setInfoStream(PrintStream) (public, IndexWriterConfig.java:426-446) whose message() can throw a RuntimeException, and that then recover from the failed commit() the documented way: rollback(), or close() with commitOnClose=false (which calls rollback(), IndexWriter.java:1363-1368). Shapes that can throw: a custom subclass writing to its own log (message() cannot throw a checked exception, so an I/O failure surfaces as UncheckedIOException); a PrintStream subclass whose println throws; Lucene's own JavaLoggingInfoStream behind a java.util.logging Handler that throws, or a backend bridged to it that forwards appender failures (Log4j 2 documents ignoreExceptions=false: "logging exceptions will be forwarded to the caller").
Default vs opt-in. Opt-in: the default InfoStream.NO_OUTPUT cannot reach the window. Everything else is default: default codec, any directory (the independent reproducer reports the same corruption on an on-disk FSDirectory; that run's output was not kept), one thread.
Scale. None: one committed document is enough, and the result is deterministic.
What is lost. The last good commit: its segment files are deleted, so DirectoryReader.open and a new IndexWriter both fail (CorruptSegmentInfoException / CorruptIndexException: a segment .si file is missing, _0.si in the repro, _1.si in a larger probe with three segments). With the default KeepOnlyLastCommitDeletionPolicy there is no older commit to fall back to. The failure is loud, at the next open.
How likely. Contrived: no shipped InfoStream or mainstream logging backend throws by default, the throw must hit exactly this line after the commit's earlier log lines succeeded, and an Error there (OutOfMemoryError included) is tragic and safe. Our searches found no report of it, although it has been in every release since 4.8.0 (2014). For these reasons the adjudication and the verifiers leaned Low; it is kept at Medium because committed data is lost.
Context, not claimed. The failed commit() also leaves a stale pending commit behind: a retried commit() finishes it, then throws NullPointerException ("fileNames" is null) and closes the writer as tragic (if a merge ran in between, the new commit references deleted files); the default close() refuses with "cannot close: prepareCommit was already called with no corresponding call to commit" (IndexWriter.java:1313-1316) although the application never called prepareCommit. A maintainer said on #9738 that continuing after a failed commit is unsupported, so these are context only.

Minimal repro

One JUnit test, one thread, public classes and methods only: a subclass of InfoStream (public, marked @lucene.internal) that throws once on "done all syncs", two empty documents, ByteBuffersDirectory. It fails on main with assertions off (the last commit cannot be opened) and on (the RefCount assertion in rollback()), and passes with the fix.

TestRollbackAfterFailedCommit.java (one test, lucene/core)61 lines

/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.apache.lucene.index;

import org.apache.lucene.document.Document;
import org.apache.lucene.store.ByteBuffersDirectory;
import org.apache.lucene.store.Directory;
import org.apache.lucene.tests.util.LuceneTestCase;
import org.apache.lucene.util.InfoStream;

public class TestRollbackAfterFailedCommit extends LuceneTestCase {

  // Expected: rollback() after a failed commit keeps the last commit (1 doc); actual: it deletes
  // that commit's files.
  public void testRollbackKeepsLastCommit() throws Exception {
    boolean[] fail = {false};
    InfoStream infoStream =
        new InfoStream() {
          @Override
          public void message(String component, String message) {
            if (fail[0] && message.startsWith("done all syncs")) {
              throw new RuntimeException("boom");
            }
          }

          @Override
          public boolean isEnabled(String component) {
            return true;
          }

          @Override
          public void close() {}
        };
    try (Directory dir = new ByteBuffersDirectory()) {
      IndexWriter w = new IndexWriter(dir, new IndexWriterConfig().setInfoStream(infoStream));
      w.addDocument(new Document());
      w.commit();
      w.addDocument(new Document());
      fail[0] = true;
      expectThrows(RuntimeException.class, w::commit);
      w.rollback();
      try (DirectoryReader r = DirectoryReader.open(dir)) {
        assertEquals(1, r.numDocs());
      }
    }
  }
}

Command

cd $LUCENE
cp $REPRO/TestRollbackAfterFailedCommit.java lucene/core/src/test/org/apache/lucene/index/
./gradlew :lucene:core:test --tests org.apache.lucene.index.TestRollbackAfterFailedCommit -Ptests.seed=DEADBEEF -Ptests.asserts=false
# with assertions on (drop -Ptests.asserts=false) it fails in rollback() with the RefCount AssertionError

Expected: rollback() returns the index to segments_1, and DirectoryReader.open sees 1 document.
Observed: CorruptSegmentInfoException: _0.si cannot be read, caused by NoSuchFileException: _0.si (assertions off); AssertionError: RefCount is 0 pre-decrement for file "_1.cfs" in rollback() (assertions on).

All files of this repro: repro/LFV-1083/

More files, environment, full output382 words

new IndexWriterConfig().setInfoStream(infoStream), a ByteBuffersDirectory, two empty documents, one thread, no package-private access (it sits in org.apache.lucene.index like the upstream tests); the InfoStream class it subclasses is public but marked @lucene.internal. The test method takes about 0.15 s. The test framework picks a random default codec; the failing path is codec-independent file reference counting, and the test and an earlier test with the same steps failed with every seed tried. It passes :lucene:core:check -x test (google-java-format, ecj lint, forbidden APIs).

The test expects commit() to throw, then rollback(), then DirectoryReader.open(dir) to see the 1 committed document. In the observed output the resource name pending_segments_1 is the file segments_1: ByteBuffersDirectory.rename keeps the entry's original name.

Regression test: this same test. It passes with the recommended fix, assertions on and off.

Environment

lucene	`main 5adff78bb168 (12.0.0-SNAPSHOT); this test also fails on the released 10.6.0 (assertions on and off) and 9.12.3 (off)`
jdk	`OpenJDK 25.0.3`
os	`Linux, aarch64`

Observed output

# gradle, main 5adff78bb168, seed DEADBEEF, assertions off
TestRollbackAfterFailedCommit > testRollbackKeepsLastCommit FAILED
    org.apache.lucene.index.CorruptSegmentInfoException: segment info file: _0.si cannot be read - it may be missing or corrupt (resource=BufferedChecksumIndexInput((clone of) ByteBuffersIndexInput (file=pending_segments_1, buffers=159 bytes, block size: 1, blocks: 1, position: 0)))
        at app//org.apache.lucene.index.SegmentInfos.parseSegmentInfos(SegmentInfos.java:417)
        ...
        at app//org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:61)
        at app//org.apache.lucene.index.TestRollbackAfterFailedCommit.testRollbackKeepsLastCommit(TestRollbackAfterFailedCommit.java:56)
        Caused by:
        java.nio.file.NoSuchFileException: _0.si
            at org.apache.lucene.store.ByteBuffersDirectory.openInput(ByteBuffersDirectory.java:209)

# gradle, assertions on
TestRollbackAfterFailedCommit > testRollbackKeepsLastCommit FAILED
    java.lang.AssertionError: ... RefCount is 0 pre-decrement for file "_1.cfs"
        at org.apache.lucene.util.FileDeleter$RefCount.decRef(FileDeleter.java:278)
        ...
        at org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:578)
        at org.apache.lucene.index.IndexWriter.rollbackInternalNoCommit(IndexWriter.java:2670)
        at org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:2590)
        at org.apache.lucene.index.IndexWriter.rollback(IndexWriter.java:2582)

# the same steps as a plain main() that lists the files (numbers only; assertions off; excerpt: flush and document-writer log lines left out)
files after commit 1: [_0.cfe, _0.cfs, _0.si, segments_1]
  log [IW] startCommit: wrote pending segments file "pending_segments_2"
  log [IW] done all syncs: [_1.cfs, _0.cfe, _0.si, _1.cfe, _1.si, _0.cfs]
commit 2 threw java.lang.RuntimeException: boom; isOpen=true tragic=null
files after failed commit 2: [_0.cfe, _0.cfs, _0.si, _1.cfe, _1.cfs, _1.si, pending_segments_2, segments_1]
  log [IW] rollback
  log [IW] rollback: done finish merges
  log [IFD] now delete 3 files: [_1.cfs, _1.cfe, _1.si]
  log [IW] rollback: infos=_0(12.0.0):c1
  log [IFD] now delete 0 files: []
  log [IFD] now delete 0 files: []
  log [IFD] now delete 3 files: [_0.cfe, _0.si, _0.cfs]
files after rollback: [segments_1]
DirectoryReader.open threw org.apache.lucene.index.CorruptSegmentInfoException, caused by java.nio.file.NoSuchFileException: _0.si

# with the fix: files after rollback: [_0.cfe, _0.cfs, _0.si, segments_1]; DirectoryReader.open: numDocs=1

Why it looks unintended

rollback() promises the index "as it was when commit() was last called"; here it deletes that commit's files.
A failed fsync at the same point is undone one statement earlier (IndexWriter.java:5993-5997); the log line after it is not.
Lucene's own testExceptionDuringRollback throws a RuntimeException from InfoStream.message (during rollback()) and asserts a good index.

The full argument379 words

rollback() promises the last commit. "This removes any temporary files that had been created, after which the state of the index will be the same as it was when commit() was last called or when this writer was first opened. This also clears a previous call to prepareCommit." (IndexWriter.java:2566-2571). prepareCommit: "After calling this you must call either commit() to finish the commit, or rollback() to revert the commit" (:3723-3725).
The writer reports its state as intact. The class javadoc reserves self-closing for "an Error, or disaster strikes during a checkpoint ... in case any internal state (buffered documents, deletions, reference counts) were corrupted" (:162-165). Here the writer stays open with getTragicException() null, and its reference counts are wrong.
The code already handles the same situation one statement earlier. A failed fsync after pendingCommitSet = true resets both fields and deletes the pending file (:5993-5997); the fix extends exactly that undo to the log line after it.
The deleter's own invariant trips. FileDeleter.RefCount.decRef asserts count > 0 ("RefCount is 0 pre-decrement", FileDeleter.java:276-282): with assertions on, rollback() fails on it.
Lucene's tests inject the same kind of fault. TestIndexWriterExceptions.testExceptionDuringRollback (LUCENE-5544) installs an InfoStream whose message() throws new RuntimeException("BOOM!") and asserts "even though we hit exception: we are closed, no locks or files held, index in good state" (TestIndexWriterExceptions.java:1980-2048). It throws during rollback(), on "rollback: done finish merges" or the test-point message "rollback before checkpoint" (:1982-1983, :1990-1991), not during commit(): the same kind of fault at a different place, which shows the tests treat a throwing InfoStream as a fault the writer must survive.
A maintainer named rollback() as the remedy for these methods. On open #9738 (2019) s1monw listed prepareCommitInternal() and startCommit() among methods whose exceptions "might cause documents go missing etc. but we treated them not as fatal or tragic events since a user could always call rollback() to go back the the last known safe-point / previous commit". This bug makes that remedy destroy the last safe point.
History. 4.7.2 survives the same injected exception (the last commit opens with 1 document); 4.8.0 does not. The difference is a side edit in LUCENE-5543 ("Remove Directory.fileExists") that its issue does not mention.

The devil's-advocate check275 words

The argument: InfoStream is a "Debugging API" marked @lucene.internal (InfoStream.java:23-29), and so is JavaLoggingInfoStream; no javadoc promises that IndexWriter survives an exception from a log callback; the JDK's ErrorManager javadoc tells a Handler to report problems "rather than throwing an Exception back to the issuer of the logging call"; mainstream backends swallow appender failures by default; an executed review by another model family (OpenAI GPT-5.6, refute lens) rated the trigger out of contract; and the same maintainer wrote on #9738 that "if somebody continues after a failed commit / prepareCommit / reopen they will end up with inconsistency and / or dataloss". This is a real argument, and a maintainer may accept only a narrow fix because of it.

Why it does not settle the question: IndexWriterConfig.setInfoStream(InfoStream) and setInfoStream(PrintStream) are public, carry no @lucene.internal mark themselves (the InfoStream class they take does, and so does PrintStreamInfoStream, which the PrintStream overload wraps) and require only a non-null argument (IndexWriterConfig.java:426-446). @lucene.internal renders as "This API is for internal purposes only and might change in incompatible ways in the next release": it warns about the API, but it does not say an implementation must not throw, and no Lucene javadoc, CHANGES or MIGRATE text says so. Lucene's own testExceptionDuringRollback injects the same kind of fault (during rollback()) and asserts a good index. The claim is about rollback(), the documented recovery that the same maintainer named, not about continuing after the failure, and no reading of any contract lets a rollback delete the previous commit. The intended-lens review of the same model family and a devil's-advocate verifier both rated it real.

Suggested fix

Recommended A: startCommit's catch undoes the pending commit when it is already set

In startCommit's catch, when pendingCommitSet is already true, reset it and pendingCommit and delete the pending segments file (toSync.rollbackCommit), as the failed-fsync path does; the existing code then releases the reference once. 7 lines. Validated: the repro passes on and off; the org.apache.lucene.index tests (1,398) and :lucene:core:check pass.

All options and trade-offs4 options · 651 words

Validated with gradle in a clean scratch worktree of 5adff78bb168 (OpenJDK 25.0.3, seed DEADBEEF, 2026-11-03): the repro fails stock with assertions on and off and passes with fix A, on and off. With fix A the closest suites TestIndexWriterExceptions (including testExceptionDuringRollback and testOnlyRollbackOnceOnException), TestIndexWriterExceptions2, TestIndexWriterCommit, TestRollback, TestTransactionRollback, TestIndexFileDeleter and TestInfoStream pass (7 classes, 72 tests, 0 failures, 5 skipped); the whole org.apache.lucene.index test package with the repro in place passes (215 classes, 1,398 tests, 0 failures, 56 skipped: nightly, monster or otherwise gated); :lucene:core:check -x test passes. The scratch tree was clean after reverting. Not run: the whole core suite, nightly or stress runs.

Why it is enough: after the undo, the existing if (!pendingCommitSet) block releases filesToCommit once and nulls it, so prepareCommitInternal's guard skips its release and rollback() finds no pending commit. rollbackCommit only deletes pending_segments_N, ignoring exceptions (SegmentInfos.java:942-956). The fix clears the stale pending commit and makes rollback() safe. It does not undo everything: startCommit's finally still records the generation (segmentInfos.updateGeneration), as after a failed fsync, so a retried commit() writes segments_3. In a plain main() with the fix, a retried commit() succeeds and stores the current 2 documents, and the default close() commits them (on stock main the retry throws NullPointerException and close() refuses).

A: startCommit's catch undoes the pending commit when it is already set (validated by the checks) (recommended)

In IndexWriter.startCommit's catch (Throwable t) (IndexWriter.java:6005), before the existing if (!pendingCommitSet) block: if pendingCommitSet is true, set it to false, set pendingCommit to null and call toSync.rollbackCommit(directory), exactly as the failed-fsync catch does (:5993-5997). 7 lines added, IndexWriter.java only.

} catch (Throwable t) {
  synchronized (this) {
    if (pendingCommitSet) {
      // the pending segments file is written and synced, but we fail before returning: undo
      // it as a failed sync does, so that filesToCommit is released exactly once, below
      pendingCommitSet = false;
      pendingCommit = null;
      toSync.rollbackCommit(directory);
    }
    if (!pendingCommitSet) {
      ... // existing: release filesToCommit once, then null it
    }
  }
  throw t;
}

Trade-offs

A commit whose files were all fsynced is discarded because a log line failed, the same outcome as a failed fsync; the application sees the same exception as today. No cost on the success path.

What the checks validated

The repro: fails stock (assertions on and off), passes with A (on and off).
With A: 7 closest suites (72 tests), the org.apache.lucene.index package (215 classes, 1,398 tests, 0 failures) and :lucene:core:check -x test pass.
A plain main() with A: the failed commit removes pending_segments_2; after rollback() the index opens with 1 document; a retried commit() (writing segments_3) or the default close() commits 2 documents.

B: prepareCommitInternal's catch releases only when no commit is pending

if (filesToCommit != null && pendingCommit == null) at IndexWriter.java:3922, leaving the reference to rollback() or finishCommit, as in 4.7.2.

Trade-offs

The writer keeps a pending commit the application never asked for: the default close() still refuses with "prepareCommit was already called", and a retried commit() finishes the stale prepared commit instead of the current state. Not validated.

C: keep a throwing InfoStream out of the window

Catch and suppress a RuntimeException from that one message (the way #14668 made onTragicEvent suppress InfoStream exceptions). Moving the line alone is not enough: a throw anywhere after pendingCommitSet = true and before startCommit returns into prepareCommitInternal's try still leaves pendingCommit set while filesToCommit is released (IndexWriter.java:3914-3929, :5961-6035).

Trade-offs

Narrowest: it removes the only known trigger but keeps the asymmetry for any statement later added after pendingCommitSet = true. Not validated.

D: treat any failure after pendingCommitSet = true as tragic

In the spirit of the unmerged patch on #9738 that made every Throwable in startCommit tragic.

Trade-offs

Closes the writer because a log line failed; heavier than needed. Not validated.

Downstream note

Relevant to products that route IndexWriter's InfoStream into their own logging layer and use rollback() as the recovery after a failed commit: a logging failure on this one line can cost them the last commit.

How it was found

A bounded-exhaustive fault-injection sweep threw one exception at each callback call that commit, prepareCommit, NRT reopen and flush make on the calling thread (15,072 cases). A retried commit() failing after a throw at startCommit's "done all syncs" line was the first signal; reading the reference counting then found the rollback() consequence.

The method in full167 words

Method: semi-formal (a bounded-exhaustive single-fault injection sweep of the real lucene-core jar, then code reading witnessed by public-API probes), on main e1faeba08bde, 2026-11-02.

The sweep threw one RuntimeException or Error at the k-th call (k = 1..N) of every InfoStream.message and isEnabled call, leaf sorter, IndexSorter, per-field format chooser, ConcurrentMergeScheduler.merge, findFullFlushMerges and IndexWriterEventListener call made on the calling thread by commit, prepareCommit, DirectoryReader.open(writer), openIfChanged, flush and flushNextBuffer, in 6 configurations: 15,072 cases with assertions off.
Its first signal: in 24 cases a retried commit() threw NullPointerException ("fileNames is null") after a fault at startCommit's "done all syncs" line. Reading the reference counting of the pending commit then showed the double release and the rollback() consequence, confirmed by public-API probes (rollback, close() with commitOnClose=false, the default close(), a retried commit, a merge in between).
The case was reduced to the one JUnit test of this page. With the fix, the sweep shows no failure at startCommit.

Prior reports, history and checks

Prior reports

No report of this defect found (GitHub issues and PRs incl. the JIRA archive, the mailing lists, downstream trackers, the web; re-checked 2026-11-03 in two independent passes and again for this doc). Related, different defects: #9738 (getTragicException misses corrupting exceptions), #6981 (a double release from another cause) and #7020 (rollback deleting files after a finishCommit failure), both fixed in 2014.

Searches5 searches

The searches below found no Lucene issue, PR, JIRA entry or list thread, and nothing outside Lucene, that describes this defect (a search cannot prove that none exists, and a few queries were throttled or rate-limited). Related prior items, each about a different defect:

#9738 (LUCENE-8692, open, 2019): getTragicException() does not reflect all corrupting exceptions (notably NoSuchFileException, from a test that deletes index files under a running writer), so Solr kept shard leadership after a failed commit. In its discussion s1monw lists startCommit() and prepareCommitInternal() among methods whose non-tragic exceptions "might cause documents go missing", with rollback() as the remedy. Different: a general design stance with no mechanism; its startCommit failure came from directory.sync, a path that does undo the pending commit. Nothing about a pending commit left set after its references are released, or rollback() deleting files of the last commit. It names rollback() as the recovery this bug breaks.
#6981 (LUCENE-5919, fixed 2014): the same "RefCount is 0 pre-decrement" double release on prepareCommitInternal's failure path, from a different cause (a decRef that failed part-way and was then retried); its fix covered that cause only. Different trigger and fix site.
#7020 (LUCENE-5958, fixed 2014): after a commit-path exception, "may end out deleting wrong files if the user calls rollback, leaving a corrupt index", at finishCommit's checkpoint, after the commit is written; fixed by making that failure tragic. Different phase and fix site.
Also read, different: #6606 (LUCENE-5543, the introducing change; its discussion does not mention the move), #4189 (LUCENE-3116, open: pendingCommit paths lack tests), #6607 (LUCENE-5544, exceptions during rollback() leak locks), #5219 (LUCENE-4147, concurrent rollback and prepareCommit), #4945 (LUCENE-3872, origin of close()'s refusal), #10516, #10005, #16625 and open PR #16643, #12751, #9264, #3610, #11757, #13558, open PR #16948, #9290 (LUCENE-8243, fixed 2018: doc-values update files deleted when addIndexes is involved); #5764 (LUCENE-4699, open, 2013, an AssertionError in IndexWriter.filesExist) predates the introducing change. Trigger context only: #14668, #14731.

What was searched, when, and what turned up
Where	When	How	Found
GitHub issues, PRs and commits (apache/lucene incl. the migrated JIRA archive); Elasticsearch, OpenSearch, Solr; dev@, java-user@, issues@, commits@; JIRA; web	2026-11-02	three scripted prior-art runs of 51 queries each (mechanism and trigger/symptom terms), 53 manual GitHub searches (RefCount is 0 pre-decrement, prepareCommit was already called, fileNames is null, decRef(pendingCommit), rollbackCommit, midStartCommit, filesToCommit, last-48-hour legs, downstream repositories), 6 JQL, 15 list and 7 web queries	only the related items above
Open PRs	2026-11-02	the changed lines of all 410 open apache/lucene PRs	19 touch IndexWriter.java, IndexFileDeleter.java, FileDeleter.java or SegmentInfos.java; none changes pendingCommit, startCommit, filesToCommit, rollbackCommit or the 'done all syncs' line; two change other lines of rollbackInternalNoCommit (#16643 moves readerPool.close() before the deleter's checkpoint, #16847 widens doc counts to long)
Re-check (two independent passes)	2026-11-03	the earlier items re-read against this defect	related but about different defects; nothing describes this one
This doc	2026-11-03	a scripted prior-art run of 45 queries (16 GitHub issue/PR, 3 commit, 2 downstream, 20 list, 4 JIRA; 719 distinct hits, the closest #9738 and #9290), 5 manual GitHub searches (3 more rate-limited, their phrases covered by the scripted run), every apache/lucene issue and PR updated since 2026-11-02 (53), the upstream commits on IndexWriter.java, IndexFileDeleter.java, FileDeleter.java and SegmentInfos.java, and 1 web query (2 more throttled)	nothing new; #9290 and #5764 read, both different defects
Before publishing	2026-11-04	the same scripted prior-art run without cache (45 queries: 16 GitHub issue/PR, 3 commit, 2 downstream, 20 list, 4 JIRA), 6 manual GitHub searches (pendingCommitSet, "done all syncs", filesToCommit and three phrase searches), the 38 apache/lucene issues and PRs updated since 2026-11-03, and 2 web queries	the same 719 hits, none new; nothing describes this defect

History

Introduced in 4.8.0 by LUCENE-5543 (2014), which moved the startCommit call into the try whose failure path releases the commit's files; 4.7.2 survives the same fault. Executed causal isolation pins it on that one hunk. Affected through 10.6.0 and main; not fixed upstream as of 2026-11-03.

Versions tested8 rows

Introduced by LUCENE-5543 "Remove Directory.fileExists" (migrated as #6606), committed 2014-03-23 by mikemccand: branch_4x 60e2a6c89b9 (svn r1580557, shipped in 4.8.0) and trunk ef3973c5890 (svn r1580540, which git blame on main still shows). Besides its topic it moved prepareCommitInternal's startCommit(toCommit) call into the try whose failure path releases filesToCommit, guarded by a new if (filesToCommit != null). startCommit nulls filesToCommit only when it fails before pendingCommitSet = true, so a later failure is now released there and again by rollback().

 4.7.2                                        4.8.0 (LUCENE-5543)
   try { maybeMerge(...); success = true; }     try { maybeMerge(...); startCommit(toCommit); success = true; }
   finally { if (!success) decRef(files) }      finally { if (!success && filesToCommit != null) decRef(files) }
   startCommit(toCommit);

The window itself is older and was harmless in 4.7.2, where only rollback() released those references: pendingCommitSet and its guard date from LUCENE-3182 (2011), the "done all syncs" line from LUCENE-4055 (2012), rollback()'s decRef(pendingCommit) from LUCENE-1347 (2008). Later rewrites kept the shape: LUCENE-5919 (2014), sequence numbers (2016), LUCENE-8217 (2018), LUCENE-8962 (2020). Every release tag from 4.8.0 to 5.5.5 contains 60e2a6c89b9 and every one from 6.0.0 to 10.6.0 contains ef3973c5890. Not fixed upstream: upstream main f72004515ec (2026-11-03); the last commit to IndexWriter.java is #16963 (5adff78bb168 itself, one log line elsewhere), and IndexFileDeleter.java, FileDeleter.java and SegmentInfos.java last changed before it.

Versions and branches executed, with results
Version or commit	Verdict	Result
4.7.2	last unaffected	passes with assertions on and off while the injected exception fires at the same line: the last commit opens with 1 document
4.8.0	first: 4.8.0	fails: `_0.si` missing (assertions off), the RefCount assertion in rollback() (on)
4.8.0 with only the LUCENE-5543 hunk reverted	bisect	passes
4.7.2 with only the LUCENE-5543 hunk applied	bisect	fails (`_0.si` missing)
4.10.4, 5.0.0, 6.6.6, 7.7.3, 8.11.4, 9.0.0, 10.0.0	affected	a version-neutral main() form of the test fails, assertions off (the last commit cannot be opened)
9.12.3, 10.5.2	affected	the JUnit test fails, assertions off
10.6.0	affected	this test fails, assertions on and off (Maven Central jars, SHA-1 checked)
main `5adff78bb168`	affected	this test fails through gradle, on and off; passes with fix A

Independent checks

An independent reproducer, two executed reviews by another model family (OpenAI GPT-5.6), a devil's-advocate verifier and an adjudication reproduced it with their own tests; one review called the trigger out of contract while agreeing on the mechanism and fix. This doc's gradle runs reproduced it on main 5adff78bb168 and validated the fix.

Checks and verdicts316 words

The checks ran on main e1faeba08bde (the cross-model reviews on ea57360912ed). Between those mains and 5adff78bb168 the only change to the files involved is one log line in prepareCommitInternal's merge-on-commit branch (#16963); startCommit, prepareCommitInternal's catch and rollbackInternalNoCommit are identical.

Each independent check and its verdict
Check	Verdict	What it found
finder	found	Fault-injection sweep signal, then code reading; its own JUnit test (`JavaLoggingInfoStream` with a throwing `java.util.logging` handler) fails on main (on, off), 10.6.0, 10.5.2 and 9.12.3; the fix makes it pass.
independent reproducer	real	Own test (a plain `InfoStream` subclass) fails on main (on, off), 10.5.2 and 9.12.3; reports the same corruption on an on-disk `FSDirectory` (that run's output was not kept) and with `close(commitOnClose=false)`; passes with the fix.
cross-model review, refute lens (OpenAI GPT-5.6, executed)	not real on trigger legality only	Reproduced it with its own test and a passing control (on, off, 3 of 3 iterations), agreed on the root cause and the fix, and ran upstream rollback and InfoStream tests with the fix (pass); ruled an exception from the `@lucene.internal` `InfoStream` out of contract.
cross-model review, intended lens (OpenAI GPT-5.6, executed)	real (high)	Own test through the public `setInfoStream(PrintStream)` overload fails on and off; its control passes; the fix passes it and the upstream tests.
devil's-advocate verifier	real (medium-high)	Built the strongest misuse case (internal debugging API, JDK and backend conventions) and rejected it for `rollback()`; reproduced on 10.6.0 and with a plain `InfoStream` subclass and a passing control.
adjudication	confirmed (high)	Re-ran the reproducer's test with a new seed: fails on and off; limited the claim to `rollback()` and `close()` with `commitOnClose=false`.
this doc (gradle)	reproduced, fix validated	The test fails on main through gradle, on and off; with fix A it passes, and the closest suites, the index test package and `:lucene:core:check -x test` pass.

Diagram file

The diagram as a still image, generated with AI and checked against the code. It is under 10 MB, the limit for a GitHub attachment.

PNG 522 KB

Sources and provenance

Finding record, Lucene and upstream sources, repro files and diagram notes1,741 words

Finding record

LFV-1083, snapshot of 2026-11-06

Registry title: If commit() fails with an InfoStream exception after the pending segments file is synced, the commit's file references are released twice, so a later rollback() (or close() with commitOnClose=false) deletes files of the last commit and the index can no longer be opened
Area: index / IndexWriter commit failure handling (IndexWriter.startCommit, prepareCommitInternal and rollback; IndexFileDeleter reference counts)
Priority: Medium, rank 114: If the opt-in InfoStream throws on one commit log line and the application then calls the documented rollback(), with assertions off the files of the last good commit are deleted, so the index can no longer be opened and committed data is lost, in every release since 4.8.0. The trigger is contrived and the failure shows up loudly at open time, but losing committed data keeps it in Medium.
Impact class: index unusable
Impact note: Hits only applications that install their own InfoStream (off by default: a custom InfoStream subclass, a PrintStream whose println throws, or the shipped JavaLoggingInfoStream behind a logging handler or bridge that passes backend failures to the caller) when it throws a RuntimeException on the one 'done all syncs' line that commit() logs after it has written and fsynced the new pending segments file, and that then call the documented recovery rollback() (or close() with commitOnClose=false): with assertions disabled, as in production, the writer deletes files that the last good commit still references, so DirectoryReader.open and a new IndexWriter fail with CorruptSegmentInfoException or CorruptIndexException (NoSuchFileException _0.si) and the committed data is gone unless the application has a backup; an Error at the same point, OutOfMemoryError included, is tragic and keeps the last commit intact; no scale is needed; every release from 4.8.0 to 10.6.0 and main.
Summary: You enable an InfoStream on an IndexWriter, commit one document, add a second one and call commit() again, and the InfoStream throws a RuntimeException on the 'done all syncs' message that Lucene logs after it has written and fsynced pending_segments_2; commit() fails without closing the writer, so you call rollback(), whose javadoc says the index will then be as it was when commit() was last called. Instead, startCommit has kept the half-finished commit as its pending commit while prepareCommitInternal's catch has already released that commit's file references, and rollback() releases them a second time, so the file deleter removes _0.cfe, _0.cfs and _0.si, which segments_1 still needs. DirectoryReader.open(dir) then fails with CorruptSegmentInfoException (NoSuchFileException _0.si), close() with commitOnClose=false does the same, and with -ea rollback() throws AssertionError 'RefCount is 0 pre-decrement' instead.
Introduced: 60e2a6c89b944e48b7c157b2729a186d2017ba9f (branch_4x, svn r1580557, first released in 4.8.0; the same change on trunk is ef3973c58900c03f765ae8581c4e2dde149339b5, svn r1580540), LUCENE-5543 'Remove Directory.fileExists' (migrated as apache/lucene#6606; svn era, no pull request; author mikemccand). Besides its topic it moved the startCommit(toCommit) call in prepareCommitInternal into the try block whose failure path releases filesToCommit (guarded by 'if (filesToCommit != null)'); startCommit nulls filesToCommit only when it fails before pendingCommitSet = true, so a failure after that point is now released there and again by rollback() [exec: single-hunk causal isolation with a patched IndexWriter: 4.8.0 with only that prepareCommit hunk reverted passes, 4.7.2 with only that hunk applied fails]. The window itself is older and was harmless before: the 'done all syncs' message after pendingCommitSet = true has its current form since LUCENE-4055 (2012) and rollback() has released pendingCommit since at least LUCENE-1347 (2008), so 4.7.2 released the references once [read]. Later rewrites kept the shape: LUCENE-5919 (2014), sequence numbers (2016), LUCENE-8217 (2018), LUCENE-8962 (2020) [read] (2014-03-23 (author date of 60e2a6c89b9 on branch_4x, 2014-03-23T18:03:55Z; trunk ef3973c5890, 2014-03-23T17:05:36Z))
First affected version: 4.8.0 [exec: a version-neutral main() probe fails with -ea and -da]; last unaffected release 4.7.2 [exec: passes with -ea and -da while the injected exception fires at the same line; 4.7.0 to 4.7.2 do not contain the change by git merge-base --is-ancestor]
Affected versions: every release from 4.8.0 to 10.6.0 (released 2026-10-29, 4ae946e5a9b) and main e1faeba08bde (unreleased 12.0.0-SNAPSHOT). [exec] the independent reproducer's JUnit TestRollbackAfterFailedCommitKeepsLastCommit fails on main with -ea and -da (run by the independent reproduction, the adjudication, the version-history check and the registration check), on 10.6.0 (-da, version-history check, released Maven jars whose core carries 4ae946e5a9b), 10.5.2 and 9.12.3 (-da, the independent reproduction); the finder's JUnit fails on main (-ea, -da), 10.6.0, 10.5.2 and 9.12.3 (-da); version-neutral main() probes fail on 4.8.0 (-ea and -da), 4.10.4, 5.0.0, 6.6.6, 7.7.3, 8.11.4, 9.0.0, 9.12.3, 10.0.0, 10.6.0 and main (-da) and pass on 4.7.2 (-ea and -da). [read] every release tag from 4.8.0 to 5.5.5 contains 60e2a6c89b9 and every one from 6.0.0 to 10.6.0 contains ef3973c5890 (git tag --contains). Not new in 10.6. Not fixed upstream: upstream main e1faeba08bd (2026-11-02) has the same startCommit, prepareCommitInternal and rollbackInternalNoCommit code; the newest upstream commit to IndexWriter.java is 169cbffd12f (2026-09-21), already in e1faeba; none of the 410 open apache/lucene PRs changes the pending-commit handling
Status: documented (brief, checks and a per-finding doc exist)
Verification: Confirmed by AI-run checks: an independent repro, a cross-model review that disputed it (the adjudication ruled it real), an intended-behaviour check, a prior-report search and a skeptical audit; not yet triaged by a person.
Found by: semi-formal: exhaustive check; also reached by code reading
Upstream and prior reports: no report of this defect found (2026-11-02, read-only; the prior-report record, on top of the finder's scripted prior-report search with 51 queries and the searches of the independent reproduction, the intended-behaviour check and the adjudication): two more scripted prior-report searches with 51 queries each (mechanism terms and trigger/symptom terms; GitHub issues, PRs and commits, Elasticsearch, OpenSearch and Solr, the dev@, java-user@ and issues@ lists, the JIRA archive, the web), 53 manual GitHub searches (phrases 'RefCount is 0 pre-decrement', 'prepareCommit was already called', 'fileNames is null', 'decRef(pendingCommit)', rollbackCommit, 'pending segments', midStartCommit, filesToCommit; last-48-hour legs; downstream repositories), the changed lines of all 410 open apache/lucene PRs (19 touch IndexWriter.java, IndexFileDeleter.java, FileDeleter.java or SegmentInfos.java; none changes pendingCommit, startCommit, filesToCommit, rollbackCommit or the 'done all syncs' line), 6 JQL queries, 15 list queries and 7 web queries. Related, all by others: open #9738 (LUCENE-8692, 2019), where a maintainer (s1monw) lists startCommit() and prepareCommitInternal() among the methods whose non-tragic exceptions 'might cause documents go missing', with rollback() as the remedy that this bug breaks, naming no mechanism; closed #6981 (LUCENE-5919, fixed 2014): the same 'RefCount is 0 pre-decrement' double release on prepareCommitInternal's failure path from a different cause (a decRef that failed part-way), fixed for that cause only; closed #7020 (LUCENE-5958, fixed 2014): the same consequence ('may end out deleting wrong files if the user calls rollback, leaving a corrupt index') at finishCommit's checkpoint, fixed by making that failure tragic. Related, different: #6606 (LUCENE-5543, the introducing change, whose discussion does not mention the move), #10516, #4189, #6607, #5219, #4945, #10005, #16625 with open PR #16643, #12751, #9264, #3610, #11757, #13558, open PR #16948, neo4j/neo4j#9121 (symptom only). Refresh (2026-11-02): none of the 15 apache/lucene PRs and 2 issues updated since 2026-10-31 and none of the 6 items created since then is related; keyword searches 'pendingCommit rollback', 'startCommit exception', 'done all syncs', 'rollback deletes files of last commit' and 'InfoStream exception commit rollback' return nothing new (only the unrelated LUCENE-2192 and LUCENE-4975); upstream main is still e1faeba08bd. Not fixed upstream.
Lucene commit tested: main 5adff78bb168 (12.0.0-SNAPSHOT), 2026-11-03: the one-test repro through gradle in a clean scratch worktree, stock (fails, assertions on and off) and with the recommended fix (passes, on and off); with the fix the closest suites (7 classes, 72 tests), the whole org.apache.lucene.index test package (215 classes, 1,398 tests, 0 failures) and :lucene:core:check -x test pass. The same test fails on the released 10.6.0 (on and off) and 9.12.3 (off). Earlier runs on main e1faeba08bde and releases back to 4.7.2 are listed under the version history.

Lucene and upstream sources

Finding record: lucene/core/src/java/org/apache/lucene/index/IndexWriter.java:2566-2571 (rollback javadoc), :2642-2648 (rollbackInternalNoCommit releases pendingCommit), :3915-3933 (prepareCommitInternal: startCommit inside the try, catch releases filesToCommit), :5974-5997 (startCommit: pending file written, pendingCommitSet and pendingCommit set, failed sync undone), :6000-6022 ('done all syncs' message and testPoint inside the try; catch releases only if !pendingCommitSet)
lucene/core/src/java/org/apache/lucene/index/IndexFileDeleter.java (decRef and checkpoint; the -ea AssertionError comes from IndexFileDeleter.decRef:620 <- checkpoint:578 in the run)
The finding in 30 seconds: lucene/core/src/java/org/apache/lucene/index/IndexWriter.java:3773-3935 (at 5adff78bb168) (prepareCommitInternal: incRef at :3857, filesToCommit at :3909, startCommit inside the try at :3910-3914, catch releasing filesToCommit at :3920-3933)
lucene/core/src/java/org/apache/lucene/index/IndexWriter.java:5911-6036 (at 5adff78bb168) (startCommit: prepareCommit at :5973, pendingCommitSet/pendingCommit at :5983-5984, sync and its undo at :5990-5998, 'done all syncs' at :6000-6002, testPoint at :6004, catch at :6005-6021, Error tragic at :6031-6033)
lucene/core/src/java/org/apache/lucene/index/IndexWriter.java:2601-2672 (at 5adff78bb168) (rollbackInternalNoCommit: decRef(pendingCommit) at :2642-2649, checkpoint/refresh/close at :2670-2672); :4271-4318 (finishCommit releases filesToCommit at :4287-4288); :6132-6137 (testPoint)
lucene/core/src/java/org/apache/lucene/index/IndexFileDeleter.java:489-500 (at 5adff78bb168) (close), :544-591 (checkpoint), :603-626 (incRef/decRef)
lucene/core/src/java/org/apache/lucene/util/FileDeleter.java:95-130 (at 5adff78bb168) (decRef, deletion at count 0), :276-282 (RefCount.decRef assertion)
lucene/core/src/java/org/apache/lucene/index/SegmentInfos.java:942-961 (at 5adff78bb168) (rollbackCommit; prepareCommit writes an invalid checksum until finishCommit), :996-1010 (finishCommit renames the pending file)
Who is affected: lucene/core/src/java/org/apache/lucene/index/IndexWriterConfig.java:426-446 (at 5adff78bb168) (setInfoStream overloads: public, 'Must not be null')
lucene/core/src/java/org/apache/lucene/index/IndexWriter.java:1312-1316 (at 5adff78bb168) (shutdown refuses with a pending commit), :1363-1368 (close: commitOnClose=false calls rollback())
Minimal repro: lucene/core/src/java/org/apache/lucene/store/ByteBuffersDirectory.java:178-192 (at 5adff78bb168) (rename keeps the FileEntry and its original name)
Why it looks unintended: lucene/core/src/java/org/apache/lucene/index/IndexWriter.java:2566-2571 (at 5adff78bb168) (rollback javadoc), :3720-3733 (prepareCommit javadoc), :162-165 (class javadoc: self-close on Error or checkpoint disaster), :5993-5997 (failed-sync undo)
lucene/core/src/java/org/apache/lucene/util/FileDeleter.java:276-282 (at 5adff78bb168) (RefCount assertion); util/InfoStream.java:23-29 (@lucene.internal debugging API)
lucene/core/src/test/org/apache/lucene/index/TestIndexWriterExceptions.java:1980-2048 (at 5adff78bb168) (testExceptionDuringRollback: throws during rollback() on 'rollback: done finish merges' or 'rollback before checkpoint', :1982-1983, :1990-1991)
lucene/core/src/java/org/apache/lucene/index/IndexWriterConfig.java:426-446 (at 5adff78bb168) (setInfoStream overloads)
https://github.com/apache/lucene/issues/9738 (s1monw's comments of 2019-03: rollback() as the remedy; continuing after a failed commit unsupported)
Lucene main 5adff78bb168 build-tools/build-infra/src/main/java/org/apache/lucene/gradle/plugins/java/RenderJavadocPlugin.java:506 (the @lucene.internal note text); lucene/core/src/java/org/apache/lucene/util/PrintStreamInfoStream.java:27 (@lucene.internal)
Suggested fix: lucene/core/src/java/org/apache/lucene/index/IndexWriter.java:3920-3933 (at 5adff78bb168) (option B site), :5993-5997 (the failed-sync undo A mirrors), :6000-6021
lucene/core/src/java/org/apache/lucene/index/SegmentInfos.java:942-956 (at 5adff78bb168) (rollbackCommit)
Prior reports: https://github.com/apache/lucene/issues/9738, https://github.com/apache/lucene/issues/6981, https://github.com/apache/lucene/issues/7020
History: GitHub commits API, read-only, 2026-11-03: apache/lucene main head f72004515ec; last commits on IndexWriter.java (5adff78bb16), IndexFileDeleter.java (ba7f659b1d8), FileDeleter.java (54283150a81), SegmentInfos.java (330ee57b167)
Independent checks: a Lucene main checkout git diff e1faeba08bde 5adff78bb168 -- IndexWriter.java (one log line at :3888)
Diagrams: lucene/core/src/java/org/apache/lucene/index/IndexWriter.java:3857, (at 5adff78bb168) :3920-3933, :5973-6021, :2642-2672

Other statements trace to the hunt's working files (models, logs, reviews and briefs), which are not published.

Repro files

TestRollbackAfterFailedCommit.java (one test, lucene/core): repro/LFV-1083/TestRollbackAfterFailedCommit.java (61 lines)

About the diagrams

A step-by-step picture of the repro, made in parallel with this doc from the code at 5adff78bb168 and the executed runs; the PNG is the summary still.

assets/LFV-1083/bug.png (522 KB)

Built 2026-11-06 from the hunt's record of this finding and its doc file. Lucene main under test in this hunt: ea57360912ed. Every listed finding was re-run on Lucene main e1faeba08bde on 2026-11-02.