A failed NRT reader open whose leaf sorter throws never releases its file reference, so the open writer keeps that point in time's segment files on disk

IndexWriterConfig#setLeafSorter (LUCENE-9507, since 8.9.0) gives the writer a Comparator<LeafReader> that orders the segments of every NRT reader opened from it. The NRT open StandardDirectoryReader.open(IndexWriter, ...), behind DirectoryReader.open(writer), openIfChanged(reader[, writer]) and SearcherManager/ReaderManager refreshes over a writer, first calls writer.incRefDeleter(segmentInfos), which makes the writer keep every segment file (not segments_N) of the point in time it is about to expose, and then constructs the reader. That constructor sorts the segment readers with the leaf sorter. If the sorter throws, the open's catch (Throwable) gives back only the segment readers; the file reference is released only by the returned reader's close(), and no reader is returned. IndexWriter.getReader treats only an Error as tragic, so the writer stays open and usable while it keeps those files referenced for the rest of its life. With two one-document segments, after the failed open, forceMerge(1) and commit(), the six files of _0 and _1 are still in the directory; 19 failed refreshes with merges in between left 118 files / 524,782 bytes in the directory for an index of 4 files / 41,211 bytes. No wrong results, no index damage, and the next IndexWriter opened on the directory deletes the files. A 9-line guarded decRefDeleter in the same catch fixes it. Every release from 8.9.0 to 10.6.0 and main is affected (8 of those releases and main executed, the rest by reading).

• NRT open: StandardDirectoryReader.open(IndexWriter, IOFunction, SegmentInfos, boolean, boolean) (StandardDirectoryReader.java:110-166 at 645d4fe14a20), called by IndexWriter.getReader (IndexWriter.java:613-615) under the writer's lock. DirectoryReader.open(writer) (DirectoryReader.java:122, :144), openIfChanged(reader) on an NRT reader (through doOpenFromWriter, getReader at StandardDirectoryReader.java:466), openIfChanged(reader, writer) (DirectoryReader.java:340, :403-408, then doOpenIfChanged(IndexWriter, boolean) at StandardDirectoryReader.java:434-442: doOpenFromWriter at :438 for the reader's own writer, getReader at :440 for another; :452 is the same call in the ExecutorService overload), and SearcherManager/ReaderManager refreshes over a writer all end there.
• Deleter reference: IndexWriter.incRefDeleter(segmentInfos) (IndexWriter.java:6306-6317) adds one reference in the writer's IndexFileDeleter to every file of the segments in a point in time, not to segments_N (it calls deleter.incRef(segmentInfos, false), IndexFileDeleter.java:603-609); decRefDeleter (:6319-6336) takes it back. IndexFileDeleter deletes a file when its count reaches zero (IndexFileDeleter.java:47-50).
• Leaf sorter: the Comparator<LeafReader> from IndexWriterConfig#setLeafSorter (IndexWriterConfig.java:517-527), null by default.

1. The NRT open gets one segment reader per segment (StandardDirectoryReader.java:136), keeps it or drops a fully deleted one (:137-145), and then calls writer.incRefDeleter(segmentInfos) (:148).
2. It returns new StandardDirectoryReader(dir, readers.toArray(...), writer, segmentInfos, writer.getConfig().getLeafSorter(), ...) (:150-157). The BaseCompositeReader constructor runs Arrays.sort(subReaders, subReadersSorter) (BaseCompositeReader.java:74-77): the application's comparator.
3. The comparator throws. The catch (StandardDirectoryReader.java:158-164) runs IOUtils.applyToAll(readers, SegmentReader::decRef) and rethrows. The only decRefDeleter for this reference is in the reader's doClose (:548-568), and no reader was returned.
4. IndexWriter.getReader's catch (IndexWriter.java:711-721) calls onTragicEvent only for an Error, so after the sorter's RuntimeException the writer stays open and usable, with a count on every segment file of that point in time that nothing will take back.
5. Later merges and commits drop their own references, but these files never reach zero while the writer lives. They are still there after IndexWriter.close(); the next IndexWriter opened on the directory deletes them as unreferenced.

Without a leaf sorter no application code runs in that constructor; its only other explicit throw is the "Too many documents" check (BaseCompositeReader.java:89-98), which IndexWriter prevents. So before 8.9.0 no realistic input reached this gap (read, not executed); the one contrived way, then and now, is a caller-supplied InfoStream (trusted configuration) whose message throws inside incRefDeleter after deleter.incRef has run (IndexWriter.java:6306-6317), which the fix below does not cover either. By reading only: IndexWriter's merge-on-refresh second open (maybeReopenMergedNRTReader, IndexWriter.java:755, :766) calls the same method and would leak the same way.

Repeated: a SearcherManager over the writer, 20 rounds of 200 documents, commit, a refresh whose sorter throws (19 failures; round 0 has one segment, so the sorter is not called), forceMerge(1) and commit. After round 19: 118 files / 524,782 bytes in the directory (a heap ByteBuffersDirectory; by reading, any Directory keeps them the same way), the same after the writer closes, and 4 files / 41,211 bytes once a new IndexWriter has opened and closed on the directory. The control without failures never has more than 10 files.

• Who is hit. NRT applications with IndexWriterConfig#setLeafSorter (for example a newest-first sorter for time-based indices) that refresh through DirectoryReader.open(writer), openIfChanged(reader[, writer]), or a SearcherManager/ReaderManager built on the writer (SearcherManager.java:66, ReaderManager.java:41; refreshIfNeeded at SearcherManager.java:194-205). Executed: DirectoryReader.open(writer) and SearcherManager.maybeRefresh().
• Default vs opt-in. Opt-in: the leaf sorter is null by default (LiveIndexWriterConfig.java:422-431), and Lucene ships none.
• What triggers it. The sorter must throw during an NRT open of two or more segments (with one segment the sort makes no comparison): a comparator bug on a segment that lacks the sort field (Lucene's own test sorter handles a null getPointValues, TestIndexWriterReader.java:1212-1222), or an I/O error inside a correct sorter, which Comparator.compare forces it to rethrow unchecked. Elasticsearch's time-series leaf sorter does exactly that (read through the GitHub API by a verifier).
• How much. One failure pins every segment file of that point in time for the writer's lifetime: after merges, up to the index size at the time of the failure. Each further failure pins the then-current segment files again, which adds up when new segments or merges are written between the attempts (118 files / 524,782 bytes against a 41,211-byte index after 19 failures). Any index size.
• When it builds up. Only if the application keeps the writer open and retries. A product that closes the writer on a refresh failure does not accumulate (a verifier cites Elasticsearch's failEngine("refresh failed ...")), and ControlledRealTimeReopenThread ends at the first failed refresh (ControlledRealTimeReopenThread.java:212-216 catches only IOException and rethrows it unchecked). A service that catches the exception and retries maybeRefresh() on its own schedule accumulates.
• What does not happen. No wrong or resurrected results (no reader or commit refers to these files), no index modification, no hang. The refresh fails loudly with the sorter's exception, and the next IndexWriter on the directory deletes the files.
• Related findings. LFV-1059: the same trigger on the non-NRT reopen factory, which leaks segment reader references; LFV-626: the keep check in this same method, before incRefDeleter, which leaks one segment reader. Each is an independent fix.

The smallest input: two segments are the fewest that make the sorter run, a sorter that always throws needs no field or data, and forceMerge(1) plus commit() is the shortest way to make _0 and _1 unreferenced. The plain ByteBuffersDirectory keeps the oracle to "which files are left". The expectThrows passes in every run, so the failing open really happened. The same test is the regression test. Assertions on and off give the same failure, and so does another seed (FC1067, run by the fact check). All runs were on Linux aarch64 with OpenJDK 25.0.3; the bug is plain control flow.

# gradle, stock 645d4fe14a20, seed DEADBEEF, assertions on
TestNrtOpenLeafSorterFailureReleasesFiles > testFailedNrtOpenDoesNotKeepFilesOfMergedAwaySegments FAILED
    java.lang.AssertionError: file of a merged-away segment was not deleted: _0.cfe
        at org.apache.lucene.index.TestNrtOpenLeafSorterFailureReleasesFiles.testFailedNrtOpenDoesNotKeepFilesOfMergedAwaySegments(TestNrtOpenLeafSorterFailureReleasesFiles.java:43)

# assertions off (-Ptests.asserts=false): the same failure
# 10.6.0 release jars, assertions on and off: the same assertion

# with the fix, assertions on and off
classes=1 tests=1 failures=0 errors=0 skipped=0

# closest suites (TestIndexWriterReader, TestDirectoryReader, TestDirectoryReaderReopen, TestIndexFileDeleter, TestNRTReaderWithThreads, TestSearcherManager), seed DEADBEEF
stock:    classes=6 tests=112 failures=0 errors=0 skipped=1
with fix: classes=6 tests=112 failures=0 errors=0 skipped=1

# an independent verifier's test, oracle 'every file belongs to the latest commit', assertions off:
# files neither in the latest commit nor held by a reader expected:<[]> but was:<[_0.cfe, _0.cfs, _0.si, _1.cfe, _1.cfs, _1.si]>

• The pair contract. IndexWriter.decRefDeleter (IndexWriter.java:6319-6324): "Record that the files referenced by this {@link SegmentInfos} are no longer in use. Only call this if you are sure you previously called {@link #incRefDeleter}."
• The same catch. LUCENE-5262 (#6326, 2013, "StandardDirectoryReader should decRef readers on exception, not close them") added the cleanup that gives the segment readers back when the open fails (LUCENE-8215, #9262, 2018, rewrote it as today's catch (Throwable)); the deleter reference added in 2014 (LUCENE-5434, #6497) was never added to it.
• The writer is meant to stay usable. getReader's catch rethrows a non-Error without a tragic event (IndexWriter.java:711-721; the inner catch says "Uh oh, tidy up then rethrow", :669-676). The class javadoc reserves self-closing for an Error or a failed checkpoint, when "reference counts" may be corrupted (:162-165).
• Tests. TestIndexWriterReader.testNRTOpenExceptions (:1108-1153): "test that several failed attempts to obtain an NRT reader don't leak file handles"; its injected failure comes before incRefDeleter, so it never reaches this gap. No test at 645d4fe14a20 makes a leaf sorter throw: the only test that sets one, testIndexReaderWriterWithLeafSorter (:1204-1222), does not throw.
• No precondition on the sorter. setLeafSorter (IndexWriterConfig.java:517-527) and getLeafSorter (LiveIndexWriterConfig.java:422-431) say nothing about exceptions, and Comparator.compare may throw.
• History. The missing release existed since 2014, but no realistic input reached it: before LUCENE-9507 (48715fe898bf, 2021) the constructor could throw only "Too many documents", which IndexWriter prevents (a throwing caller-supplied InfoStream inside incRefDeleter is the only, contrived, exception). LUCENE-9507 put the application's comparator inside it; none of the review comments on #32, apache/lucene-solr#2256 or #2473 discuss exceptions.

The strongest case for "intended" (the devil's-advocate verifier and the intended-lens reviewer): Lucene tolerates leftover files, since the doClose comment calls them "harmless" because "the next time IW is opened on the index, it will delete them" (StandardDirectoryReader.java:554-559), LUCENE-5574 (#6636) turned the test framework's unreferenced-files check off by default, and on #12228 rmuir wrote that a new IndexWriter "will delete the unnecessary files" and that IndexWriter should not "try to be a superhero and do dangerous things such as deleting files when handling exceptions"; the sorter is application code and the repro's sorter always throws; DirectoryReader.open(writer) is @lucene.experimental and incRefDeleter/decRefDeleter are @lucene.internal.

Why it fails: the tolerance covers a writer that is already closed or a tragic event, while here the writer stays open after a non-Error exception and keeps the files for its whole life; the fix only gives the count back: in the repro it deletes nothing while handling the exception (the directory listing is unchanged right after the failed open), and by reading the writer itself still references those files at that moment, so it does not do what that comment warns about. An always-throwing comparator is within Comparator's contract, no javadoc asks for a non-throwing sorter, and a correct sorter doing I/O reaches the same path. Experimental is about API stability, not a licence to leak, and the user path is public. Every reviewer rated the intended case weak; it explains the Low priority, not the leak.

The hunt's fix, unchanged, run through gradle in a scratch worktree of 645d4fe14a20 (OpenJDK 25.0.3, seed DEADBEEF) with the minimal repro and the closest suites. A broader patch that also covers the sibling leaf-sorter leaks was validated by the hunt.

In StandardDirectoryReader.open(IndexWriter, ...) (StandardDirectoryReader.java:126-165), declare boolean deleterIncRefd = false before the try and set it to true right after writer.incRefDeleter(segmentInfos) (:148); in the catch, after the segment readers are given back, call writer.decRefDeleter(segmentInfos) when the flag is set and add any exception from it as suppressed. +9 lines.

+    boolean deleterIncRefd = false; // declared before the try
       writer.incRefDeleter(segmentInfos);
+      deleterIncRefd = true;
       return new StandardDirectoryReader(...); // runs the leaf sorter
     } catch (Throwable t) {
       // ... decRef the segment readers (unchanged) ...
+      if (deleterIncRefd) {
+        try {
+          writer.decRefDeleter(segmentInfos);
+        } catch (Throwable t1) {
+          t.addSuppressed(t1);
+        }
+      }
       throw t;

Trade-offs

• Why the flag. The catch also runs when readerFunction.apply or the keep check fails, before incRefDeleter. An unconditional decRefDeleter would then release a reference this open never took, one that belongs to a commit point or another reader: in the TLA+ model the unguarded variant violates NoOverRelease for every failure before incRefDeleter.
• No deletion during exception handling. It only gives the count back. The open runs under the writer's lock on a clone of the writer's current segments, which the writer itself still references at that moment (by reading; with the fix, the fact check's probe saw the directory listing unchanged right after the failed open), so the files go only at the next merge or commit that drops them.
• Writer closed meanwhile. decRefDeleter calls ensureOpen(); its AlreadyClosedException would be added as suppressed and the sorter's exception rethrown.
• Not covered. It does not make incRefDeleter itself transactional: if incRefDeleter throws after deleter.incRef has run (only a throwing caller-supplied InfoStream can do that; trusted configuration, contrived), the flag is still false and those references stay, as they do today.
• Scope. No change on success. Fixes this leg only; the reopen-factory leak (LFV-1059) and the keep-check leak (LFV-626) are other lines.

What the checks validated

Through gradle in a scratch worktree of 645d4fe14a20 (OpenJDK 25.0.3, seed DEADBEEF, 2026-11-02): the minimal test fails on stock and passes with the fix, assertions on and off. The closest suites (TestIndexWriterReader, which holds testNRTOpenExceptions and testIndexReaderWriterWithLeafSorter, TestDirectoryReader, TestDirectoryReaderReopen, TestIndexFileDeleter, TestNRTReaderWithThreads, TestSearcherManager: 6 classes, 112 tests) pass on stock and with the fix (1 skipped both times). Earlier, the hunt's shadow checks, an independent verifier's own guarded fix and the refute reviewer's own test plus testIndexReaderWriterWithLeafSorter passed with the same change.

The guarded release above, plus the StandardDirectoryReader constructor sorts a copy (super(directory, readers.clone(), leafSorter), so the initial open's cleanup still sees every reader once), plus LFV-1059's try/catch around the reopen factory's constructor call.

Trade-offs

One patch closes the family, but it is three changes needing three tests; the copy costs one array copy per open.

What the checks validated

By the hunt, not re-run here: it passes this test, LFV-1059's minimal test, the initial-open late-sort test and 99 upstream tests (TestDirectoryReader, TestDirectoryReaderReopen, TestIndexWriterReader, TestIndexFileDeleter, TestNRTReaderWithThreads; standalone JUnit runner, seed 5EED0018).

The same call without the flag.

Trade-offs

Over-releases when the failure comes before incRefDeleter (a failing readerFunction or keep check): it would take back a reference owned by a commit point or another reader. The TLA+ model shows the violation.

Move incRefDeleter after the constructor.

Trade-offs

The returned reader's doClose always calls decRefDeleter, so a failing incRefDeleter (for example ensureOpen on a writer closed meanwhile) would need its own cleanup that closes the reader without that call: more invasive than the guarded release for the same effect.

Method: semi-formal, a hypothesis formed while modelling, then witnessed by an executed test.

• A hunt unit (2026-10-26), building a TLA+ model of IndexReader reference counting (ReaderLifecycle.tla), noticed this leg by code reading while mapping the model to StandardDirectoryReader.open(IndexWriter, ...) and witnessed it with a public-API JUnit (testNrtOpenLeaksFileReferences), the second manifestation of its leaf-sorter finding. LFV-1059 covers that finding's reopen leg and leaves this leg out of its claim; a later probe re-ran it.
• The adjudication (2026-11-01) wrote a TLA+ model of the NRT open's acquire/release protocol (NrtOpen.tla) with a fault at five selected steps (each of two readerFunction.apply calls, the second segment's keep check, incRefDeleter failing before its increment, and the constructor; not every statement that can throw) and checked 35 TLC configurations: only the constructor fault violates NoDeleterLeak (10 distinct states), only the keep-check fault violates NoSegmentReaderLeak (that is LFV-626), each fix alone leaves the other violation, and an unguarded decRefDeleter violates NoOverRelease. It then reduced the witness to the one-test repro.