MultiIndexMergeScheduler is a 10.3.0 New Feature, but the class is package-private, so applications cannot construct it through the public API

Lucene 10.3.0 lists MultiIndexMergeScheduler under New Features as "a production multi-tenant merge scheduler", and the release announcement advertises it. Its javadoc tells users to create one instance per index and use it like a lone ConcurrentMergeScheduler, and its public constructor is "The main MultiIndexMergeScheduler constructor -- use this one". But the class itself is declared class MultiIndexMergeScheduler extends MergeScheduler, without public, so application code in any other package cannot import it or call that constructor through the public API: javac reports "MultiIndexMergeScheduler is not public in org.apache.lucene.index; cannot be accessed from outside package". It has been so in every release that has the class (10.3.0 to 10.6.0) and on main. It is a documentation/API-visibility mismatch, not a runtime defect: the failure is a loud compile error, the feature is opt-in and experimental, and nothing returns wrong results or damages an index. Adding public alone is not enough, because the shared scheduler behind every instance is never given its merge and thread limits, so the documented usage would then stall at the first merge.

• MultiIndexMergeScheduler.java:36: class MultiIndexMergeScheduler extends MergeScheduler {, no access modifier. The package is exported (module-info.java:39, exports org.apache.lucene.index;), but Java still hides a package-private type from every other package.
• The members are written as public API: the "use this one" constructor (:41-42), getDirectory() (:55-57), getCombinedMergeScheduler() (:59-61), setInfoStream(InfoStream) (:87-89) and CombinedMergeScheduler.peekSingleton() (:126-128). Only the test constructor (:48-49, "for unit testing") is package-private on purpose. The nested CombinedMergeScheduler (:100) is package-private too, although the public getter returns it.
• The javadoc (:30-32): "The multi-tenant merge scheduling can be used easily by creating a MultiIndexMergeScheduler instance for each index and then using each instance normally, the same way you would use a lone ConcurrentMergeScheduler."
• The only test, TestMultiIndexMergeScheduler, is in package org.apache.lucene.index; (line 17), so it compiles and the visibility gap stays invisible to Lucene's build. It also configures the shared scheduler by hand (:47, :121), so it never meets the first-merge stall either.
• Lucene renders javadoc with -protected (RenderJavadocPlugin.java:464), so the class has no page in the published API docs (the 10.3.0 page returns 404 while the ConcurrentMergeScheduler page returns 200, checked 2026-11-01); the CHANGES entry (lucene/CHANGES.txt:1581, under "Lucene 10.3.0" / "New Features", lines 1555 and 1579) and the release announcement are the user-visible promise.

• javac: 2 errors, "MultiIndexMergeScheduler is not public in org.apache.lucene.index; cannot be accessed from outside package", at the import and at new MultiIndexMergeScheduler(dir).
• Plain reflection through the public constructor: IllegalAccessException ("cannot access a member of class org.apache.lucene.index.MultiIndexMergeScheduler with modifiers \"public\"").
• javap on the lucene-core jars of 10.3.0 to 10.5.2, 10.6.0 and main prints class org.apache.lucene.index.MultiIndexMergeScheduler (no public) with a public (Directory) constructor.
• Precision: two unsupported routes construct it, both only on the class path: code an application places inside org.apache.lucene.index (a split package), and deep reflection (setAccessible(true) on the public constructor). Both fail on the module path, because lucene-core is a named module that owns the package and does not open it: javac reports "package exists in another module", and setAccessible throws InaccessibleObjectException. So it cannot be used through supported means.

• Who is hit. Anyone who wants what #15015 delivers, multi-tenant merge scheduling (many indexes in one JVM sharing one pool of merge threads; the original request is #13883, "A multi-tenant ConcurrentMergeScheduler"), and writes the code the javadoc describes.
• Opt-in. The default merge scheduler is ConcurrentMergeScheduler (LiveIndexWriterConfig.java:137), and the class is @lucene.experimental (MultiIndexMergeScheduler.java:34).
• Scale. Unusable through supported means in all 8 releases that ship it, 10.3.0 (2025-09-13) to 10.6.0 (2026-10-29), and on main. No runtime harm: no wrong results, hang, data loss or index damage.
• How likely. Every developer who tries it hits it at once, but the class has no published javadoc page and GitHub code search finds no user outside Lucene's own test, so few have realistically tried.

One test class, one method, deterministic, single thread. It uses Lucene's public API plus Lucene's test framework (LuceneTestCase, newDirectory()), so it is a test in Lucene's core test sources, not a standalone application; standalone application code fails the same way, on the class path and as a named module. The failure is the compile error itself, so there is no runtime assertion; the result does not depend on the seed or on assertions (compilation fails before any test runs).

Placing the file under lucene/core/src/test/org/example/ lets gradle compile it with Lucene's own test setup while keeping it outside org.apache.lucene.index. google-java-format (./gradlew tidy) wraps its long comment line into two.

Regression test: this same test. It fails to compile on stock main and passes with the fix. If committers choose to stop advertising the class instead, no test applies.

mkdir -p $LUCENE/lucene/core/src/test/org/example
cp $REPRO/TestMultiIndexMergeSchedulerFromApplication.java $LUCENE/lucene/core/src/test/org/example/
cd $LUCENE && ./gradlew :lucene:core:test --tests org.example.TestMultiIndexMergeSchedulerFromApplication -Ptests.seed=DEADBEEF -Ptests.asserts=true
# same failure with -Ptests.asserts=false

# gradle, main 645d4fe14a20, seed DEADBEEF, -Ptests.asserts=true (identical with false)
> Task :lucene:core:compileTestJava
$LUCENE/lucene/core/src/test/org/example/TestMultiIndexMergeSchedulerFromApplication.java:5: error: MultiIndexMergeScheduler is not public in org.apache.lucene.index; cannot be accessed from outside package
import org.apache.lucene.index.MultiIndexMergeScheduler;
                              ^
$LUCENE/lucene/core/src/test/org/example/TestMultiIndexMergeSchedulerFromApplication.java:14: error: MultiIndexMergeScheduler is not public in org.apache.lucene.index; cannot be accessed from outside package
        MergeScheduler scheduler = new MultiIndexMergeScheduler(dir)) {
                                       ^
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
2 errors

> Task :lucene:core:compileTestJava FAILED
BUILD FAILED

# with the recommended fix (and with `public class` alone): 1 test, 0 failures, both ways

# plain reflection through the public constructor (the finder's and the independent reproduction's tests, stock main):
java.lang.IllegalAccessException: class ... cannot access a member of class org.apache.lucene.index.MultiIndexMergeScheduler with modifiers "public"
# setAccessible(true) first: constructs it on the class path; with lucene-core on the module path:
java.lang.reflect.InaccessibleObjectException: Unable to make public org.apache.lucene.index.MultiIndexMergeScheduler(org.apache.lucene.store.Directory) accessible: module org.apache.lucene.core does not "opens org.apache.lucene.index" to unnamed module @...

• The promise. lucene/CHANGES.txt:1581 (10.3.0, New Features): "GITHUB#15015: MultiIndexMergeScheduler: a production multi-tenant merge scheduler". The 10.3.0 announcement on dev@ and java-user@: "Adds a MultiIndexMergeScheduler – a multi-tenant wrapper that allows sharing a common merge scheduler across multiple instances". Javadoc :30-32 and :41 as quoted above.
• Deliberate narrowing went only to the second constructor. The reviewer (vigyasharma) wrote: "make it pkg private for testing (not exactly safe but makes the intent explicit)" (review comment 2262089375), and PR commit 7a6ee719bb ("package private alternate constructor") did exactly that; the main constructor stayed public.
• The review assumes users reach the class (indirect evidence). The quotes below are all replies in one review thread, rooted at vigyasharma's "Should we make this class private static?" (2248892958) on the nested CombinedMergeScheduler; they discuss how users configure the shared scheduler behind a MultiIndexMergeScheduler, which presumes users hold one, and none is about the top-level modifier. vigyasharma: "users get access to the configuration setters" (2250080260), "I think it is okay to let users directly work with public APIs for the underlying CMS" (2253329820), "It's fine to keep it public too. We probably need it to let people create index groups that share a CMS?" (2257747066, about the nested CombinedMergeScheduler, in the thread on whether to make it private static). The author (corecursion): "the user needs access to the shared CMS for calling configuration functions" (2248960412). No comment asks to hide the top-level class.
• History. Package-private from the PR's first commit ad8cf7b6aa (2025-07-30) through all 17 PR commits and the merge cfb407bb0080; the declaration was never edited since.
• Precedent. #10750 (LUCENE-9711, open), "gradle precommit no longer catches accidental package-private APIs?": its example was a new API "accidentally package private, missing its public modifier", hidden because its tests live in the same package.

The strongest case for "not a bug" (rated moderate by the cross-model reviews): a package-private type is outside the public API by definition; its javadoc is not even published; it is @lucene.experimental; and made public as written, it would stall at the first merge, so maybe it was kept hidden until ready.

Why it fails: CHANGES and the release announcement, which are published, present it as a usable "production" feature, so the published promise and the code disagree whatever the intent. Nothing in the 17 PR commits or the review records a decision to hide the class, and the review discusses what users will call. @lucene.experimental renders as "WARNING: This API is experimental and might change in incompatible ways in the next release." (RenderJavadocPlugin.java:502), a stability warning for public API, not a visibility marker. The stall argues about how to fix it, not whether the mismatch is intended. The cross-model reviews said "not real" with low confidence; neither could run code. The works-as-designed review called its own verdict procedural and agreed on the root cause; the refute review's rested on its inability to run code and on the point that a package-private type is not public API, the case answered here. Neither disputed the root cause.

Three variants were run through gradle in a scratch worktree of 645d4fe14a20 (OpenJDK 25.0.3, --offline, seed DEADBEEF): public class alone; plus default limits for the shared scheduler; plus javadoc for the three public methods, which the build's missing-javadoc check demands once the class is public. Removing or rewording the advertisement was not run. Which direction to take is the committers' call.

MultiIndexMergeScheduler.java +10/-1: public on line 36; default merge and thread limits when the JVM-wide CombinedMergeScheduler is created; javadoc on getDirectory, getCombinedMergeScheduler and setInfoStream (the wording is a suggestion).

@@ -33,7 +33,7 @@ import org.apache.lucene.util.InfoStream;
  *
  * @lucene.experimental
  */
-class MultiIndexMergeScheduler extends MergeScheduler {
+public class MultiIndexMergeScheduler extends MergeScheduler {
   private final Directory directory;
   private final CombinedMergeScheduler combinedMergeScheduler;
   private final boolean manageSingleton;
@@ -52,10 +52,15 @@ class MultiIndexMergeScheduler extends MergeScheduler {
     this.manageSingleton = false;
   }
 
+  /** Returns the index directory whose merges this scheduler forwards to the shared scheduler. */
   public Directory getDirectory() {
     return directory;
   }
 
+  /**
+   * Returns the shared ConcurrentMergeScheduler that runs the merges of every index, for example to
+   * change its merge and thread limits.
+   */
   public CombinedMergeScheduler getCombinedMergeScheduler() {
     return combinedMergeScheduler;
   }
@@ -84,6 +89,7 @@ class MultiIndexMergeScheduler extends MergeScheduler {
   // We created this method because we cannot easily override the initialize() method
   // in ConcurrentMergeScheduler. We don't need the initDynamicDefaults() part in the
   // initialize() method, and only need the setInfoStream().
+  /** Sets the InfoStream of the shared scheduler. */
   public void setInfoStream(InfoStream infoStream) {
     this.combinedMergeScheduler.setInfoStream(infoStream);
   }
@@ -107,6 +113,9 @@ class MultiIndexMergeScheduler extends MergeScheduler {
     private static synchronized CombinedMergeScheduler acquireSingleton() {
       if (singleton == null) {
         singleton = new CombinedMergeScheduler();
+        // IndexWriter never calls initialize() on the shared scheduler, so resolve the
+        // AUTO_DETECT merge and thread limits here, as initialize() does for a lone CMS.
+        singleton.setDefaultMaxMergesAndThreads(false);
       }
       singletonRefCount++;
       return singleton;

Trade-offs

• Why the second hunk. IndexWriter calls initialize() only on the scheduler it was given (IndexWriter.java:960); for this class that is MergeScheduler.initialize (MergeScheduler.java:79), which only stores the info stream; the source comment at MultiIndexMergeScheduler.java:84-86 says "We don't need the initDynamicDefaults() part". The shared scheduler is created with a plain new (MultiIndexMergeScheduler.java:107-113), so maxMergeCount stays -1 (ConcurrentMergeScheduler.java:66, 88), and maybeStall loops while a merge is pending and mergeThreadCount() >= maxMergeCount (ConcurrentMergeScheduler.java:611), which 0 >= -1 always satisfies; the indexing thread waits until a merge is no longer pending or someone changes the shared scheduler's limits. A lone CMS avoids that through initialize -> initDynamicDefaults (ConcurrentMergeScheduler.java:533-535, :442-444). Lucene's test configures the shared scheduler by hand (TestMultiIndexMergeScheduler.java:47, 121), and the #15015 author noted in review that it needs that setup (corecursion, comment 2248960412). The review settled on that hand configuration through the getter (2253329820); corecursion also asked whether the shared scheduler "should come with reasonable defaults that are usable for most people" (2253039480), which is what this hunk does. It is one reasonable design: documenting the hand setup, or forwarding initialize(), would also close the gap.
• Why the javadocs. Once the class is public, :lucene:core:renderJavadoc fails with 3 errors, "MultiIndexMergeScheduler.getDirectory (method): javadocs are missing" and the same for getCombinedMergeScheduler and setInfoStream.
• setDefaultMaxMergesAndThreads(false) picks the SSD defaults once per shared scheduler (maxThreadCount = max(1, cores / 2), maxMergeCount = maxThreadCount + 5, ConcurrentMergeScheduler.java:166-192); the singleton is dropped when the last instance closes (MultiIndexMergeScheduler.java:115-124) and the next one gets the defaults again, so an explicit setMaxMergesAndThreads made earlier is lost with it; on a live singleton an explicit setMaxMergesAndThreads still overrides the defaults. The formula is the code's: the AUTO_DETECT_MERGES_AND_THREADS javadoc (ConcurrentMergeScheduler.java:62-64) says max(1, min(4, cpuCoreCount/2)), but the code (:190) has no cap of 4.
• This hunk does not create the shared scheduler's intra-merge executor, which initialize() also does (ConcurrentMergeScheduler.java:536-538); that looks harmless, because MultiIndexMergeScheduler does not override getIntraMergeExecutor, so IndexWriter (IndexWriter.java:3558, 5542) gets MergeScheduler's same-thread executor (MergeScheduler.java:38, 65-67). Alternatives, not run: forward initialize() from MultiIndexMergeScheduler to the shared scheduler (same package, so it can override the package-private method; it would also pass on IndexWriter's info stream and create that executor; whether to forward getIntraMergeExecutor too is a separate choice); and return ConcurrentMergeScheduler from getCombinedMergeScheduler(), as the reviewer first proposed (comment 2250080260). Today's return type is package-private, so mims.getCombinedMergeScheduler().setDefaultMaxMergesAndThreads(false) does not compile in application code even with public class ("is defined in an inaccessible class or interface"); only assigning the result to a ConcurrentMergeScheduler variable first does.
• A public class gets a published javadoc page and becomes API that back-compat reviews cover.

What the checks validated

• Repro, TestMultiIndexMergeScheduler and the merging check, assertions on and off: 3 suites, 5 tests, 0 failures each way.
• Merging check (not the repro): an earlier same-package test that indexes 20 documents through the "use this one" constructor with maxBufferedDocs(2) and a LogDocMergePolicy with merge factor 2, failing if addDocument is stuck for 10 s: passes (0.78 s in the assertions-off run; the assertions-on log does not list its time; 0.58 s without the javadocs, assertions on).
• Lint: :lucene:core:checkGoogleJavaFormat, ecjLintMain, ecjLintTest, forbiddenApisMain, forbiddenApisTest and renderJavadoc pass, with the repro's long comment line wrapped as google-java-format wants.

The one-keyword change on line 36.

Trade-offs

The repro compiles, but the documented usage then stalls: the merging check fails with "addDocument stalled for 10 s in the merge scheduler; shared scheduler was CombinedMergeScheduler: maxThreadCount=-1, maxMergeCount=-1". An application could avoid it only by an undocumented step (assign getCombinedMergeScheduler() to a ConcurrentMergeScheduler variable, then call setDefaultMaxMergesAndThreads(false)). renderJavadoc also fails on the 3 undocumented public methods.

What the checks validated

Repro and TestMultiIndexMergeScheduler: 4 tests, 0 failures, assertions on and off. Merging check: fails (stall, as above). Lint, run with the shared-defaults change added: renderJavadoc fails (3 missing javadocs); format, ECJ and forbidden-apis pass.

Remove the class and its test, or keep it package-private and reword the javadoc so it no longer tells users to create instances, with a CHANGES note that the 10.3.0 feature is not available through the public API.

Trade-offs

Drops an announced feature that, as far as GitHub code search shows, nobody outside Lucene uses. The released 10.3.0 CHANGES text cannot be changed. Others are removing unused classes right now (#16873 merged on main, #16862 and #16880 open), so committers may prefer this. Not run.

Method: non-formal. A probe ("advertised fix or feature unreachable from the public API") scanned 37 release jars and 3,951 classes, found 7 non-public classes that nothing outside their own tests references, and 2 of them advertised. A hunt unit (a sweep of never-named public API in core and queries) took this one.

• The finder decided it by code reading (the class modifier, the javadoc, CHANGES 10.3.0), javap of the 10.3.0 and 10.5.2 jars and the #15015 history, then a JUnit test from another package (2 of 2 fail).
• The independent reproduction wrote an application-side test that compiles application source with javac, with a ConcurrentMergeScheduler control; the intended-behaviour check argued the works-as-designed side and ran class-path, module and split-package probes; an adjudication confirmed it (high) as a documentation/API-visibility mismatch.
• 2026-10-29 the repro was reduced to one public-API test; 2026-11-01 this doc re-ran it through gradle and validated the fix.