A reader opened with the expert lower-minimum DirectoryReader.open forgets that minimum, so isCurrent(), openIfChanged() and maybeRefresh() throw on an unchanged index

DirectoryReader.open(IndexCommit, int minSupportedMajorVersion, Comparator<LeafReader>) (expert, since 9.0) opens an index whose segments come from a major below Version.MIN_SUPPORTED_MAJOR, which the default open refuses. The open range-checks the minimum and passes it to the one call that reads the commit; the reader does not keep it. isCurrent() and the reopen path later re-read the commit through the SegmentInfos overloads that apply the default Version.MIN_SUPPORTED_MAJOR, so the per-segment version gate rejects the very commit the reader already has open. With nothing committed since the open, isCurrent(), openIfChanged(reader) and SearcherManager / ReaderManager.maybeRefresh() all throw IndexFormatTooOldException on every call. The open reader keeps serving its commit (no wrong results, no data loss); to see a newer commit, the application has to call the expert open again (the old reader can stay open until the new one replaces it). The overload's other parameter, leafSorter, is carried to reopened readers; the minimum is not. Keeping the minimum on the reader and passing it to the two re-reads fixes it. Every release from 9.0.0 through 10.6.0 is affected, and so are branch_11x and main.

1. DirectoryReader.open(commit, minSupportedMajorVersion, leafSorter) (DirectoryReader.java:189-194) calls StandardDirectoryReader.open(directory, minSupportedMajorVersion, commit, leafSorter, null) (StandardDirectoryReader.java:77-108). It range-checks the minimum (:87-93), passes it to the one call that reads the commit, SegmentInfos.readCommit(directory, segmentFileName, minSupportedMajorVersion) (:94-95), and builds the reader with new StandardDirectoryReader(directory, readers, null, sis, leafSorter, false, false) (:100-101). The reader has no field for the minimum (:45-48): it is dropped here.
2. isCurrent() (:529-544), for a reader without a writer, re-reads the latest commit with SegmentInfos.readLatestCommit(directory) (:537), which is readLatestCommit(directory, Version.MIN_SUPPORTED_MAJOR) (SegmentInfos.java:581-583).
3. openIfChanged(reader) goes to doOpenNoWriter(null, ...), which calls isCurrent() first (StandardDirectoryReader.java:477-482). SearcherManager.refreshIfNeeded (SearcherManager.java:194-206) and ReaderManager.refreshIfNeeded (ReaderManager.java:92-94) call openIfChanged.
4. A reopen onto a different commit goes through doOpenFromCommit (StandardDirectoryReader.java:496-505), which reads it with SegmentInfos.readCommit(directory, segmentFileName) (:501), also the default minimum (SegmentInfos.java:293-296). openIfChanged(reader, newerCommit) reaches it directly; openIfChanged(reader) reaches it only after isCurrent() (StandardDirectoryReader.java:479-482), which already throws while the latest commit is one the default gate refuses.
5. The per-segment gate (SegmentInfos.java:523-545) throws IndexFormatTooOldException when a segment's minVersion major is below the minimum it is given (:529). With the default 9 it rejects the commit the reader opened with 8.
6. openIfChanged(reader, reader.getIndexCommit()) does return null: it short-circuits on the segments file name before reading anything (StandardDirectoryReader.java:487-490).
7. The overload's other parameter survives: doOpenIfChanged(SegmentInfos, ...) passes subReadersSorter, the leafSorter, to the reopened reader (:507-511, from LUCENE-9507). There is no minimum to pass.

Lucene's unsupported.8.11.4-cfs.zip back-compat fixture, upgraded by the stock 9.12.3 IndexUpgrader: created major 8, one segment written by 9.12.3 with minVersion 8.11.4, 35 documents. On main 645d4fe14a20 the default open throws and open(commit, 8, null) opens 35 documents; then isCurrent(), openIfChanged(reader) and SearcherManager.maybeRefresh() throw "Index has segments derived from Lucene version 8.x and is not supported by Lucene 11.0.0. This Lucene version only supports indexes with major version 9 or later (found: 8, minimum supported: 9)" from StandardDirectoryReader.isCurrent:537. With the fix all three return the contract values.

• Who is hit. Applications that keep an archived or read-only older-major index searchable after an upgrade, open it with DirectoryReader.open(commit, min, leafSorter) (or its ExecutorService variant), and then call isCurrent(), openIfChanged(), or refresh it through a SearcherManager / ReaderManager. On 11.0 and main (minimum 9) the demonstrated case is an index created by 8.x whose segments were all rewritten by 9.x or 10.x, for example by IndexUpgrader; 8.x's default codec, Lucene87, is no longer shipped there (an application that supplies an older codec itself could reach it with other old indexes too, by reasoning). The line splits at 10.4.0, where #14607 moved the default gate from the commit's created major to each segment's minVersion: on 10.4-10.6 the index is one with a segment derived from 8.x (an 8.x-created index whose segments all have minVersion 9 or later opens by default there and is not hit); on 10.0-10.3 it is any index created by 8.x; on 9.x any index created by 7.x.
• Opt-in. Default opens and NRT readers are not affected; the way in is the expert open (the public but @lucene.internal StandardDirectoryReader.open(Directory, SegmentInfos, ...), given a SegmentInfos read with a lower minimum, builds the same kind of reader, StandardDirectoryReader.java:168-184).
• Scale. None: one document is enough, and every call fails, with assertions on and off.
• Loud, no data damage. A clean IOException on every refresh. The open reader keeps serving exactly its commit: no wrong results, no lost data, no file changes. A probe of 1,500 lifecycle calls on an unchanged index and 901 after a newer commit found no file change, no open input, no ref-count drift, no lock retention, and a newer commit was never reported as current.
• Why nobody reported it, probably (an inference). The known users of the API open such readers once and never refresh them: Elasticsearch's ReadOnlyEngine (refresh() returns NO_REFRESH), OpenSearch's earlier ReadOnlyEngine (OpenSearch#5429) and a proposed Pinot fallback (apache/pinot#19894, an open PR).
• Workaround. Call open(commit, min, leafSorter) again on the latest commit, then close the old reader (executed by the devil's advocate with the old reader still open).

• Contract. openIfChanged (DirectoryReader.java:228-229): "If the index has changed since the provided reader was opened, open and return a new reader; else, return null"; @return null if there are no changes (:242). isCurrent (:628-629): "If this reader was created by calling open, then this method checks if any further commits (see IndexWriter#commit) have occurred in the directory." The expert overload is one of those open methods; neither contract excludes it.
• The expert javadoc (DirectoryReader.java:169-177) says the method "allows to open indices that were created with a Lucene version older than MIN_SUPPORTED_MAJOR" and "All backwards compatibility aside from the file format is optional and applied on a best effort basis." The failure is the file-format version gate itself, on the commit that already opened.
• Intent. The introducing commit c1ae6dc07c9 (LUCENE-9669): "adds an API on DirectoryReader to specify a minimum index version on a per reader basis. This doesn't apply for the IndexWriter which will fail on opening older indices." CHANGES 9.0 (lucene/CHANGES.txt:5015-5020): such indexes open "in read-only mode", and "Opening an index created with a major version of N-2 with an IndexWriter is not supported."
• Sibling parameter. leafSorter, from the same overload, is passed to reopened readers (StandardDirectoryReader.java:510), and that is tested with this very overload: in TestIndexWriterReader.testIndexReaderWriterWithLeafSorter, "Test4" opens DirectoryReader.open(latestCommit, Version.MIN_SUPPORTED_MAJOR, leafSorter) and checks the leaf order after openIfChanged (TestIndexWriterReader.java:1305-1328; open at :1311, reopen and check at :1324-1325). It is the only existing test that reopens a reader from the expert open, and it passes the default minimum, so it cannot catch this bug.
• Tests. No test reopens or calls isCurrent on a lower-minimum reader: TestBasicBackwardsCompatibility.testFailOpenOldIndex opens and closes (:856-871); TestDirectoryReader.testOpenWithInvalidMinCompatVersion (:1195-1207) and testOpenCommitWithMinVersionAndExecutor (:1267-1286) open current indexes and close them.
• History. Before c1ae6dc07c9 every open used the same default gate as the two re-reads (last rewritten by c4bf601b68f, LUCENE-5969, 2014), so a re-read could not reject a reader's own commit. That commit added the minimum to the initial open only; nobody in its PR (21 inline review comments), the issue or the dev@ RFC thread discussed reopen, isCurrent or refresh.

The strongest works-as-designed case (the devil's advocate built it and rejected it; the cross-model reviews rated it moderate): the overload is "Expert" and meant for one-shot read-only access; it promises only file-format compatibility and makes the rest "best effort"; the minimum has no getter and is not documented as a property of the reader; isCurrent and openIfChanged declare IOException; the main downstream user never refreshes; and a workaround exists (open again).

Why it fails (our reading of the sources): "best effort" reads as covering semantics on top of the file format (the design thread's examples are analysis and the encoding of norms), not the format gate (no document says so in as many words, and none says the minimum is kept across reopen either); read-only was defined as "you cannot open an IndexWriter on it"; the lifecycle contracts do not exclude these readers, and the failure needs no change to the index at all; the introducing commit calls the minimum per reader; the overload's other parameter survives reopen; and a stock released Lucene produces the trigger state. Verdict: a defect. Residual risk: a maintainer may prefer to document the limit instead. Upstream, it reads best as a reader-lifecycle inconsistency (a wrong error on an unchanged index), not a crash.

One fix was built and validated with gradle; a smaller variant was validated by the hunt; documenting the limit is the alternative.

A final field set through the package-private constructor. The expert open passes its argument; the NRT path and the public @lucene.internal open(Directory, SegmentInfos, ...) (used by NRT replication) pass Version.MIN_SUPPORTED_MAJOR; the reopen path (doOpenIfChanged(SegmentInfos, ...)) passes the reader's own value to the new reader through a private overload; isCurrent() and doOpenFromCommit use it. +29/-7, in StandardDirectoryReader.java only. The core of it:

  // the minimum major version this reader was opened with: isCurrent() and reopen use it too
  private final int minSupportedMajorVersion;

  // isCurrent()
  SegmentInfos sis = SegmentInfos.readLatestCommit(directory, minSupportedMajorVersion);

  // doOpenFromCommit
  final SegmentInfos infos =
      SegmentInfos.readCommit(directory, segmentFileName, minSupportedMajorVersion);

  // doOpenIfChanged(SegmentInfos, ...): carry it to the new reader, like subReadersSorter
  return StandardDirectoryReader.open(directory, infos, getSequentialSubReaders(),
      subReadersSorter, executorService, minSupportedMajorVersion);

Trade-offs

• One int per reader. No change for readers opened with the default minimum (the field is then Version.MIN_SUPPORTED_MAJOR, exactly what the re-reads use today) or for NRT readers.
• A reader opened with a stricter minimum (above the default) now also applies it on refresh, which is what a fresh open(commit, min, sorter) of the new commit would do; that is a behaviour change maintainers may want to decide on explicitly.
• No getter is added; one could be, but nothing needs it.

What the checks validated

• Gradle on 645d4fe14a20, fresh scratch worktrees, seed DEADBEEF: stock, the test fails with assertions on and off. With the fix: the test plus TestDirectoryReader, TestDirectoryReaderReopen, TestIndexWriterReader (incl. the leaf-sorter reopen test) and TestSearcherManager, 5 suites, 101 tests, 0 failures, 1 skipped; the test with assertions off passes.
• With the fix, every lucene/core test: 748 test classes, 8,898 tests, 0 failures, 322 skipped. TestBasicBackwardsCompatibility (lucene/backward-codecs, which opens each older back-compat index with the expert open): 816 tests, 0 failures, 2 skipped. lucene/replicator, the other caller of the public open(Directory, SegmentInfos, ...), compiles, but its 15 tests are nightly or nested-only and were skipped.
• :lucene:core:checkGoogleJavaFormat passes on the fix and the test.
• Standalone runner: the test passes with the fix (-ea, -da); the independent reproducer's 7-test lifecycle class (SearcherManager, openIfChanged(reader, newerCommit), isCurrent after a newer commit) goes from 5 failures to 0; its real-index test on the 9.12.3-upgraded 8.11.4 index from 3 failures of 4 to 0.

The finder's sketch (+14/-6) and the independent reproducer's (+16/-6): a non-final private int minSupportedMajorVersion = Version.MIN_SUPPORTED_MAJOR; assigned after new StandardDirectoryReader(...) in the expert open and after open(...) in doOpenIfChanged (the finder with a cast, the reproducer with an instanceof check), plus the same two re-read changes.

Trade-offs

Same behaviour and a smaller diff, but the field is written after construction and needs a cast or an instanceof check; the recommended option keeps it final.

What the checks validated

The finder's 2 tests, the independent reproducer's 7 synthetic and 4 real-index tests, the reviewers' own tests, TestDirectoryReader (34/34) and TestDirectoryReaderReopen (30/30) pass with it (standalone runner, ea57360912ed), the one-test repro passes on 645d4fe14a20 with assertions on and off, and a 9.0.0 backport passes on 9.0.0.

Say in the expert javadoc that a reader opened with a lower minimum cannot be refreshed and must be re-opened with the same call.

Trade-offs

Leaves isCurrent() throwing on an unchanged index, against its own contract, and SearcherManager / ReaderManager unusable for such readers.

Method: semi-formal, bounded-exhaustive translation validation (hunt unit "backcompat 9.x policy", main ea57360912ed, 2026-10-25).

• A harness synthesized 336 commits (6 created majors x 7 segment versions x 8 minVersion values including null) and ran 28 operations on each against the stock main jar, comparing every outcome with a truth table derived from the source.
• The operations "isCurrent and openIfChanged on a reader opened with a lower minimum" mismatched on all 21 structurally valid tuples whose segments' minVersion major is below MIN_SUPPORTED_MAJOR.
• Those reopen operations were added to the harness from code reading of StandardDirectoryReader while modelling the gate: the hunt chose to look there, and the harness showed that every such reader fails. A deterministic JUnit followed, later reduced to the one-test repro.