A rolled-back NRT reader can call different index content "current"

Keep a near-real-time (NRT) reader after its IndexWriter is rolled back, open a new IndexWriter on the same directory and commit: the old reader's isCurrent() can return true, although it still shows 3 documents that were never committed and none of the new ones. The new writer hands out the same SegmentInfos version numbers again, and once the reader's writer is closed, isCurrent() compares only that number. It needs no special configuration (it also fails with a plain new IndexWriterConfig()), and every executed release since 3.1.0 reproduces it; silent and transient, no corruption. A small extra check in StandardDirectoryReader.isCurrent() fixes it and passes all of :lucene:core:test (one seed; nightly tests not run).

You index with an IndexWriter and search through a near-real-time (NRT) reader that you refresh from the writer, so new documents become searchable before they are committed. Then the writer ends without committing: you call rollback() to discard a failed batch, or close() with setCommitOnClose(false), or the writer hits an I/O error during a refresh and closes itself. Lucene allows you to keep using the last reader ("Once the writer is closed, any outstanding readers may continue to be used"). Later a new IndexWriter opens the same directory (plainly, or on an explicit commit point from DirectoryReader.listCommits) and commits new documents. To decide whether to switch to a fresh reader, you ask the old one isCurrent().

It should say false: new commits have happened, and the old reader shows documents that were rolled back. Sometimes it says true.

• Version: a counter in SegmentInfos that "counts how often the index has been changed". The writer adds to it on every change and stores it in each commit. An NRT reader remembers the writer's counter at the moment it was opened, including changes that were never committed.
• Commit (segments_N): each commit is a new file with a larger N. It is the only place the counter is saved.

At step 6 the old reader shows 4 documents, 3 of which no longer exist, and misses W2-1 and W2-2. It reports itself up to date because writer 2's second commit landed on version 10, the number writer 1 had given the reader for changes it then threw away.

1. An NRT reader carries an uncommitted version. It clones the writer's live SegmentInfos (StandardDirectoryReader.java:126). SegmentInfos.changed() does version++ (SegmentInfos.java:1106-1107); the writer calls it when it names a new segment (IndexWriter.java:2179), on other changes (:2923-2925) and once more on a commit with changes (:3828-3835). These are the bumps this repro hits; others exist, e.g. deleteAll (:2799), setLiveCommitData (:4171) and advanceSegmentInfosVersion (:1417-1423).
2. Rollback keeps no record of it. Rollback restores the committed segment list (IndexWriter.java:2655) and closes the writer; nothing on disk remembers version 10. close() with commitOnClose=false calls rollback() (:1363-1368); a tragic error closes the writer through rollbackInternal() (:6098-6105).
3. A new writer counts again from the last commit. It reads the latest commit (IndexWriter.java:1085) and keeps its version, 4, so its commits climb through the same range again (7, 10, 13, ...). Opening on an explicit commit point (setIndexCommit) reads the same latest commit and keeps its version plus one (:1077-1104); its "preserve write-once" comment (:1088-1092) is about newer commits on disk, not about versions a discarded session gave only to NRT readers, and it collides too (the fact check's probe: k = 2, 5 and 8). Only opening from an NRT reader of the old writer, i.e. on the orphan's own commit point, copies the old writer's version over "so we remain write-once" (:1062-1071).
4. The closed-writer check compares only the number. isCurrent() (StandardDirectoryReader.java:529-544): if the reader's writer is closed (:531), it reads the latest commit and returns sis.getVersion() == segmentInfos.getVersion() (:540). It never asks which commit: the reader's SegmentInfos still names segments_1, the latest commit is segments_3.

In these runs each add-and-refresh moves the version by 2 and each add-and-commit by 3. The old reader sits at 4 + 2k after k refreshes, and writer 2's commits land on 7, 10, 13, ... In a sweep over k = 1 to 8 with 8 one-document commits, k = 3 (version 10) and k = 6 (version 16) collide; the other six values never do. Other workloads step differently: in the study, a writer-2 commit that only set commit user data landed on the old reader's version 4 too.

• Who is exposed. Applications that keep an NRT reader after its writer ends without committing, open a new IndexWriter on the same directory, and ask the old reader isCurrent() to decide when to open a fresh one.
• Three triggers, all public API, all executed: rollback() (ByteBuffersDirectory); close() with setCommitOnClose(false) (FSDirectory); and a tragic event: an IOException ("simulated disk full") from createOutput during an NRT refresh makes the writer close itself (isOpen() false, getTragicException() set), with no rollback call by the application. Writer 2 may be opened plainly or on an explicit commit point from DirectoryReader.listCommits (executed in the fact check); only opening it on the old reader's own commit point avoids the collision.
• No opt-in. The minimal repro fixes NoMergePolicy to keep the numbers stable; a plain new IndexWriterConfig() (default merge policy and scheduler) gives the same wrong answer at version 10 (second test). No scale needed: one document per commit.
• Silent. No exception and no log line. The application is told that a reader showing 3 never-committed documents, and missing the newly committed ones, is current, so it keeps serving that view.
• Limits. The wrong answer lasts only while the latest commit's version equals the old reader's; the next commit ends it. It needs that exact match (2 of 8 values of k in the sweep); how often real workloads align was not measured. Nothing on disk is wrong. The old reader cannot be reopened anyway (AlreadyClosedException, documented), so the application must open a new reader. One that knows a rollback or tragic close happened can do that unconditionally; one that asks the old reader isCurrent() to decide gets the wrong answer.
• Reach through Lucene's helpers. SearcherManager refreshes with openIfChanged, not isCurrent(), and its isSearcherCurrent() is package-private (SearcherManager.java:232). FilterDirectoryReader passes isCurrent() through (FilterDirectoryReader.java:152-153), so wrapped readers are affected.
• Same root, another symptom (study only). Two readers of different writer sessions can share a version with different content; the study showed SearcherLifetimeManager.record() then throwing IllegalArgumentException ("the provided searcher has the same underlying reader version yet the searcher instance differs from before", SearcherLifetimeManager.java:156-158). Not re-run in this hunt.
• Released. Every release since 3.1.0 (2011) contains the change, as far as the history clone's tags show; the 16 releases that were run all reproduce.

# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
    => java.lang.AssertionError: isCurrent() expected false, was true; reader: maxDoc 4, version 10; latest commit: maxDoc 3, version 10
       org.junit.Assert.fail(Assert.java:89)
       org.junit.Assert.assertTrue(Assert.java:42)
       org.junit.Assert.assertFalse(Assert.java:65)
       org.apache.lucene.index.TestOrphanedNrtReaderIsCurrent.testIsCurrentAfterRollbackAndNewWriterCommits(TestOrphanedNrtReaderIsCurrent.java:58)

• The javadoc. DirectoryReader#isCurrent (DirectoryReader.java:631-635): for a near real-time reader "this method checks if either a new commit has occurred, or any new uncommitted changes have taken place via the writer". Two new commits occurred. It has no caveat for readers whose writer has closed, and adds (:637-638) "if this returns false, you should call openIfChanged to get a new reader that sees the changes".
• Keeping the reader is supported. DirectoryReader.java:330-332: "Once the writer is closed, any outstanding readers may continue to be used. However, if you attempt to reopen any of those readers, you'll hit an AlreadyClosedException."
• The docs are not airtight here either way. getVersion() "returns the version recorded in the commit that the reader opened" (DirectoryReader.java:620), which does not literally describe an NRT reader's uncommitted version, and the advice to call openIfChanged when isCurrent() is false cannot work for an orphan (it throws). Neither text says that such a reader may be reported current after new commits.
• Lucene's own test expects a kept NRT reader to stay non-current across a new session. TestIndexWriterReader.testUpdateDocument (:132-200) keeps NRT reader r2 after writer.close() and asserts assertFalse(r2.isCurrent()) after that close (:177, :181), then again after a new IndexWriter adds a document (:189) and commits on close (:194). This is indirect: r2 is already non-current before the new writer opens. Before LUCENE-6523 (2015), which made a commit itself a change for NRT readers, :177, :181 and :189 were assertTrue and :194 already asserted false once the new writer had committed a new document: a direct check that a new writer's commit makes a kept reader non-current. testIsCurrent asserts false after the writer's close (:232-233). All of this covers close-with-commit only, where versions keep increasing.
• Lucene's replication layer treats version reuse as invalid. PrimaryNode.java:116-119 (replicator): "We forcefully advance the SIS version to an unused future version. This is necessary if the previous primary crashed and we are starting up on an "older" index, else versions can be illegally reused but show different results." It works around this with the @lucene.internal advanceSegmentInfosVersion (IndexWriter.java:1412-1423), for replication only. The comment is about the replication protocol, which needs unique versions, not a ruling on core SegmentInfos.version. In core, only opening a writer from an NRT reader carries the old writer's version over (:1062-1071).
• History. The closed-writer branch came with LUCENE-1976 (#3051, 052e1f5630d, 2009), an issue titled "isCurrent() and getVersion() on an NRT reader are broken". LUCENE-2805 (#3879, 6c9052d2311, 2010) turned the bug on while aiming for a correct isCurrent() inside one writer session; its discussion does not mention rollback or a new session. The testIsCurrent that LUCENE-2802 added in 2010 asserted true for an NRT reader after a commit-on-close that committed exactly what it showed; LUCENE-6523 changed that to false in 2015. Under either meaning of "current" the orphan should be non-current: its content differs and new commits occurred.
• Effect size. The answer is a boolean and it is wrong, for a reader 3 of whose 4 documents were rolled back.

The strongest case for "intended" came from one of the study's verifiers (P-model), and the cross-model reviewer rated the same case "moderate": versions are only promised to advance ("This version is advanced every time a change is made with IndexWriter", DirectoryReader.java:617-623), never to be unique across writer sessions; LUCENE-3672 (#4746, 2012) removed, on purpose, the timestamp that a new index's version started from; callers that need uniqueness, such as replication, are expected to force the version forward; opening NRT readers is @lucene.experimental; and an old reader cannot be reopened, so its isCurrent() is only a hint.

Why it fails: this report does not ask for unique versions; it is about the documented answer of isCurrent(), which has no such caveat. Keeping the reader is explicitly allowed, and Lucene's own test expects a kept reader to stay non-current across a new session. The timestamp seed never made versions unique across writer sessions: it applied only to a new index, and 3.1.0 to 3.6.2, which still have it, reproduce this bug with timestamp-sized versions. @lucene.experimental is about API stability, not about wrong answers. P-model verifier itself reproduced the isCurrent() consequence and called it a javadoc violation; it classed only the version scheme as by design. The validated fix needs no unique versions. Verdict: not intended, a defect of low-to-moderate severity.

One fix was applied in a scratch worktree of ea57360912ed and validated with the repro and the module's tests. The other options are ideas with their trade-offs; they were not run.

For an NRT reader (writer != null) whose writer is closed, require the latest commit's segments file name to equal the one in the reader's SegmentInfos, in addition to equal versions. Readers opened from a commit keep today's check. Same logic as the candidate's sketch, reformatted for google-java-format.

// we loaded SegmentInfos from the directory
if (sis.getVersion() != segmentInfos.getVersion()) {
  return false;
}
// An NRT reader can hold an uncommitted version. If its writer ended without committing, a
// later writer starts again from the last commit's version and can commit that same number
// with different content, so also require that no newer commit exists:
return writer == null || sis.getSegmentsFileName().equals(segmentInfos.getSegmentsFileName());

Trade-offs

• One file, no format change; it costs one string comparison. It can only turn true into false, and only when a commit newer than the reader's exists, which the javadoc counts as a change (since LUCENE-6523 a commit is a change for an NRT reader even when the content is the same), so those false answers are right by contract.
• Using the segments file name as the commit's identity has a precedent in the same class: doOpenNoWriter treats an equal getSegmentsFileName() as "no change" (StandardDirectoryReader.java:487-489). The reopen path of commit readers, which calls isCurrent() (:480), is unchanged.
• It does not stop the version reuse itself, so other code that treats "same version" as "same state" (the SearcherLifetimeManager case) is not covered.
• It relies on an NRT reader's SegmentInfos naming the last commit its writer saw. That holds in every run here, including the control with a legitimately current reader of a closed writer, but it was not proven for every path.
• The segments file name comes from the commit generation (SegmentInfos.java:250-253), which keeps growing in a directory that Lucene manages; if the directory is erased or replaced outside Lucene, a name could repeat. No focused test was run for an NRT reader without an initial commit, prepareCommit or OpenMode.CREATE; the explicit-commit-point path was run (below).

What the checks validated

• Before the patch (gradle, clean worktree): the repro 2/2 fail (seed DEADBEEF); with the second test added, 7 tests, 5 failures (seed 7E1EB4EA), the 2 controls pass.
• After: the repro plus 14 closest suites (TestIndexWriterReader, TestDirectoryReader, TestDirectoryReaderReopen, TestRollback, TestIndexWriterCommit, TestIndexWriterFromReader, TestNRTReaderWithThreads, TestStressNRT, TestIndexWriterExceptions, TestTragicIndexWriterDeadlock, TestIndexWriter, TestIndexWriterOnDiskFull, TestSearcherManager, TestControlledRealTimeReopenThread): 303 tests, 0 failures, 8 skipped (seed DEADBEEF).
• After: all of :lucene:core:test with both repro tests included, seed 5EED1234 (one seed, nightly tests not run): 749 classes, 8,872 tests, 0 failures, 0 errors, 320 skipped. :lucene:replicator:test ran too, but all 15 of its tests were skipped (14 are @Nightly; TestSimpleServer runs only as a nested suite), so it adds no evidence.
• After: the fact check's explicit-commit-point probe (setIndexCommit on the latest commit), which collides on unpatched main, passes with the patched class (-ea, seed DEADBEEF).
• :lucene:core:checkGoogleJavaFormat passes; the patch applies cleanly to ea57360912ed (applied with git apply; git apply --check also passes).
• Earlier, independently: the study ran the same change against 13 existing Lucene test classes plus its own test class (300 tests OK), and the second adjudication ran it as a shadow class against both tests with -ea and -da, all green.

Make a new IndexWriter start above every version a discarded session handed out. Not tried.

Trade-offs

Would cover every consumer of the version, not only isCurrent(). But a new writer cannot learn those versions: rollback() leaves the index "the same as it was when commit() was last called" (IndexWriter.java:2566-2571) and writes nothing. Seeding each new session from the clock or a random offset would be a new mechanism: the clock seed that LUCENE-3672 removed in 2012 applied only to new indexes and did not prevent this bug in 3.x (3.1.0 to 3.6.2 reproduce). Needs a maintainer's view.

After a rollback or a tragic close, open a new reader from the directory or the new writer instead of asking the old reader isCurrent(); it cannot be reopened anyway. A replication-style alternative is to call advanceSegmentInfosVersion(oldReader.getVersion()) on the new writer, as PrimaryNode does. Not executed.

Trade-offs

No Lucene change, but every application has to know about it; advanceSegmentInfosVersion is @lucene.internal.

Add a caveat to isCurrent() for readers whose writer ended without committing.

Trade-offs

Leaves the wrong answer in place; the weakest option.

Method: an LLM proof-obligation audit (semi-formal), in run 1 of the study (2026-10-22, main 7cecf79523). The study's contamination note lists this finding among those its task text did not hint at.

• An AI tool wrote down invariants of the NRT bookkeeping and discharged one obligation per mutator by hand. Invariant I4: after the writer is closed, a reader whose version equals the latest commit's version shows that commit's content. Obligation O-X7, "new IndexWriter(dir) after rollback", failed: the new writer restarts numbering at the last commit's version, below versions already shown to the old session's NRT readers. The AI tool wrote a Java repro; Z3 and KeY were used only afterwards, to mechanise the argument.
• Model-based randomized testing found the same failure later, without formal tools.
• TLA+ model checking (TLC) re-found it blind as a counterexample of property IsCurrentSound (an 11-state trace: write, NRT open, rollback, new writer, commit at the reader's version), and the Alloy solver re-found it (finding E, property ClosedWriterReaderCurrentSound, 8 steps), both translated into failing Java tests.
• Study verifiers then checked it: 8 classed it a new bug, 1 classed the version scheme itself as intended (P-model verifier, which still reproduced the isCurrent() consequence), and 1 as the same root cause in a variant. In this hunt it was re-run on main and 10.5.2, reviewed by an independent reviewer from another model family (OpenAI GPT-5.6), tie-broken, bisected and searched for prior reports; this page was then fact-checked twice and its diagram checked once.