Overlapping forceMerge calls share one target: forceMerge(1) can return with 2 segments

IndexWriter keeps one forced-merge state for the whole writer: a segmentsToMerge map and an int mergeMaxNumSegments. Every forceMerge(n) call resets that state and writes its own n into every pending and running merge, including the merges an earlier call that is still waiting started. A finished forced merge re-plans with its own, now overwritten, target, and a waiting caller returns as soon as no forced merge is pending or running; it never compares the result with its own n.

So forceMerge(1), overlapped by forceMerge(2) on another thread while it still needs another merge round, returns normally with 2 segments (3 with an overlapping forceMerge(3)). No document is lost, searches are correct and CheckIndex is clean. It needs the first call to take more than one merge round: LogByteSizeMergePolicy or LogDocMergePolicy with more than mergeFactor segments, or the default TieredMergePolicy while a natural merge runs (shown with floorSegmentMB lowered so the tiny test segments behave like real ones; with the default floor the same tiny-segment run gave 1 segment); the default policy on a static index of small segments is not affected. Introduced by LUCENE-982 (2007, first released in 2.3.0); every release from 2.3.0 to 10.6.0 and main is affected. Keeping the stricter target while an earlier call is still active fixes the repro and passes the closest suites.

1. One forced-merge state per writer. segmentsToMerge ("used by forceMerge to note those needing merging") and mergeMaxNumSegments are plain fields of the writer (:377-379).
2. Each call overwrites it. forceMerge(int, boolean) (:2237), under the writer's lock (:2249), clears and refills segmentsToMerge (:2251-2255), sets mergeMaxNumSegments = maxNumSegments (:2256) and then, under the comment "Now mark all pending & running merges for forced merge" (:2258-2259), sets merge.maxNumSegments = maxNumSegments on every pending (:2261) and running (:2271) merge. The merge that an earlier, still-waiting forceMerge(1) started now carries the target 2.
3. New merges take the latest target. maybeMerge(..., MergeTrigger.EXPLICIT, maxNumSegments) (:2281) asks findForcedMerges with this call's target and stamps each returned merge with it (:2489); a merge registered later over a segment in segmentsToMerge gets mergeMaxNumSegments (:5199-5200).
4. The cascade uses the overwritten target. A committed forced merge adds its new segment to segmentsToMerge (:5020-5024), and the merge thread re-plans with updatePendingMerges(mergePolicy, MergeTrigger.MERGE_FINISHED, merge.maxNumSegments) (:5105). With target 2 and segments _0(1), _3(2) left, LogMergePolicy returns no merge.
5. The waiter checks "any forced merge left", not its own target. The wait loop (:2284-2312) waits while maxNumSegmentsMergesPending() (:2305) is true; that method (:2326-2336) only asks whether a pending or running merge has maxNumSegments set. Once the last forced merge is done, forceMerge(1) returns normally. Nothing compares the segment count with its n.

• Who is hit. Code that runs blocking forceMerge calls with different targets on one writer from two threads, for example a "merge to 1" job and a "merge to at most N" maintenance task sharing a writer, when the later call has the larger target and starts while the first call's forced merges are still pending or running.
• Default vs opt-in. Stock components, no opt-in feature, no scale (3 one-document segments are enough). The first call must need more than one merge round: LogByteSizeMergePolicy or LogDocMergePolicy (stock, not the default) with more than mergeFactor segments; with the default TieredMergePolicy it was reproduced only when forceMerge(1) starts while a natural merge over some segments runs (executed with floorSegmentMB lowered so tiny test segments behave like real ones). A forceMerge(1) with TieredMergePolicy on a static index of small segments merges in one round and was not affected; larger TieredMergePolicy indexes were not tested.
• What goes wrong. forceMerge(1) reports success with 2 (or 3) segments, so code that relies on one segment afterwards is wrong. A segment that was not re-merged keeps its deleted documents on disk until a later merge. In an executed probe deleted documents stayed invisible to search, every live document was found once with its own stored value, CheckIndex was clean, and a second forceMerge(1) reached 1 segment with no deletions.
• Likelihood: rare. forceMerge is usually run once by one task on a static index, and no user report was found in the nearly 19 years since 2.3.0 (2008).

Two threads, because concurrency is the bug; deterministic. LogDocMergePolicy (stock, not the default) is subclassed only to turn natural merges off (findMerges returns null) and to signal when each forceMerge call has planned (findForcedMerges). The second thread starts forceMerge(2) once the main thread has planned its first round; IndexWriter plans under its own lock, so the merge _1 + _2 is registered by then (pending or already running; the reset overwrites both). The merge thread holds that merge until forceMerge(2) has planned, i.e. until after its reset. The segment count is read on an NRT reader right after forceMerge(1) returns. Ten random-seed iterations failed 10/10 with the same message.

As the regression test it is a good basis: google-java-format wants the comment above the assertion wrapped (./gradlew tidy does it); ecj and forbidden-APIs lint pass. The latch waits and join() are unbounded, so a fix that made forceMerge(2) wait for forceMerge(1) would hang the test instead of failing it, and an exception in the second thread is not asserted on the main thread.

TestOverlappingForceMerge > testForceMergeOneOverlappedByForceMergeTwo FAILED
    java.lang.AssertionError: segments when forceMerge(1) returned expected:<1> but was:<2>
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:2905AE9AAA72FA49]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.failNotEquals(Assert.java:835)
        at org.junit.Assert.assertEquals(Assert.java:647)
        at org.apache.lucene.index.TestOverlappingForceMerge.testForceMergeOneOverlappedByForceMergeTwo(TestOverlappingForceMerge.java:95)

:lucene:core:test (FAILURE): 1 test, 1  failure

# the same with -Ptests.asserts=false. The same schedule traced step by step (main, assertions on; off gives the same):
findForcedMerges by A maxSegmentCount=1 infos=[_0(1), _1(1), _2(1)] ... -> [[_1, _2]]
B calls forceMerge(2)
doMerge released segments=[_1, _2] merge.maxNumSegments=2 mergeMaxNumSegments=2
findForcedMerges by merge-thread maxSegmentCount=2 infos=[_0(1), _3(2)] ... -> null
A forceMerge(1) returned segments=[_0(1), _3(2)]
B forceMerge(2) returned segments=[_0(1), _3(2)]
RESULT forceMerge(1) returned with 2 segment(s), expected 1
# two overlapping forceMerge(1) calls instead: _0 + _3 -> _4, both return with [_4(3)]

• The contract is per call. "Forces merge policy to merge segments until there are <= maxNumSegments" (IndexWriter.java:2188); "This call will merge those segments present in the index when the call started" (:2219); "@param maxNumSegments maximum number of segments left in the index after merging finishes" (:2223). Class javadoc: "completely thread safe, meaning multiple threads can call any of its methods, concurrently" (:167-168). In the repro no segment is added after either call starts.
• No serial order explains it. One call after the other, in either order, leaves 1 segment when forceMerge(1) returns, and the executed 2-then-1 overlap ends with 1 for both calls.
• Documented interference is spelled out. forceMerge says segments flushed by other threads during the call are not merged (:2219-2221); deleteAll says concurrent forceMerge callers "may receive MergePolicy.MergeAbortedExceptions" (:2743-2745). Nothing says forceMerge calls loosen each other.
• Lucene's tests overlap forced merges. TestThreadedForceMerge calls forceMerge(1, false) and forceMerge(1) from several threads (TestThreadedForceMerge.java:92, :103), always with one target; TestIndexWriter.testRandomOperations calls writer.forceMerge(1 + random().nextInt(10), random().nextBoolean()) from several threads (TestIndexWriter.java:4290) and checks only that nothing throws.
• A committer called a related silent shortfall "a little dangerous" (TieredMergePolicy.java:857-861, LUCENE-10093: a second forceMerge's newly flushed segments are skipped: a different defect, and a hedged question rather than a settled API reading, but it treats a second forceMerge while one runs as something users do).
• History. The overwrite comes from LUCENE-982 (2007), which added a per-call target to a mechanism built when every optimize() targeted 1 segment; its discussion (#2058) never mentions overlapping calls, and no javadoc, comment, CHANGES.txt or MIGRATE.md entry says the last call's target wins.

The strongest case for "works as designed", made by the cross-model reviewer, the intended-behaviour devil's advocate and the unit's first finder (who judged it not reportable): the forced-merge state is writer-wide by design, every call deliberately marks "all pending & running merges for forced merge" with its own target, so the last call wins and applications should run force merges one at a time. The thread-safety note itself goes on: "If your application requires external synchronization, you should not synchronize on the IndexWriter instance ... use your own (non-Lucene) objects instead" (IndexWriter.java:168-170). There is a precedent: in LUCENE-1335 (#2412, 2008) committers agreed that IndexWriter makes no API promises about what concurrent commit, close, rollback, optimize or addIndexes calls mean, "except that the methods all work correctly, IndexWriter won't throw an errant exception, and your index won't become corrupt", and that "higher level synchronization by the user is the right way to ensure/enforce an ordering".

Why it fails: LUCENE-1335 is about the order of different operations, not two targets, and says nothing like "last call wins"; here every serial order gives 1 segment, so the 2-segment result is not the outcome of any ordering (running the calls one at a time does avoid it: that is the workaround, not an explanation); nothing documents the behaviour and it contradicts the per-call @param; and forceMerge(1) returns normally, i.e. reports that it finished.

One fix was built and validated with gradle on 645d4fe14a20; the alternatives come from the independent reproducer, the adjudication and the brief. A real fix must also decide what a later doWait=false call means for callers that are already waiting.

Count the forceMerge calls that have reset the shared state and not yet returned. A new call that finds one active takes min(maxNumSegments, mergeMaxNumSegments) before the reset; the rest of forceMerge(int, boolean) moves into a helper wrapped in try/finally that decrements the counter.

synchronized (this) {
  // Overlapping forceMerge calls share segmentsToMerge and mergeMaxNumSegments: keep the
  // strictest target while another call is still waiting, so this call does not loosen it.
  if (activeForceMerges > 0) {
    maxNumSegments = Math.min(maxNumSegments, mergeMaxNumSegments);
  }
  activeForceMerges++;
  resetMergeExceptions();
  ...
}
try {
  forceMergeAndWait(maxNumSegments, doWait);
} finally {
  synchronized (this) { activeForceMerges--; }
}

Trade-offs

• The later, looser call merges as far as the earlier call's target: forceMerge(2) overlapping a forceMerge(1) ends with 1 segment. That meets its own contract ("at most 2") and is what the reverse order already does.
• A doWait=false call counts only while it is inside forceMerge, which it leaves right after planning, so a non-blocking forceMerge(1) followed by a blocking forceMerge(2) still loosens the background merges. No caller waits on them, so no return breaks its contract, but maintainers should decide what that should mean.
• The waiter still does not check its own result: a policy that cannot reach n behaves as before, and segments flushed after the call started stay excluded, as documented.
• For review (read, not run): mergeMaxNumSegments is not restored when the strictest call returns, so a third call that starts while only the looser call is active also inherits the stricter target; and activeForceMerges++ sits before the try, so an Error later in that synchronized block would leave the counter raised.
• Merge-failure reporting is untouched: a later call's resetMergeExceptions() (:2250, :3039-3041) still clears mergeExceptions and bumps mergeGen, so a failure of a merge an earlier call registered can be dropped (the mechanism of the separate bug LFV-590).
• Cost: one int field and two synchronized updates per call.

What the checks validated

• Scratch worktree of 645d4fe14a20, OpenJDK 25.0.3, seed DEADBEEF: stock, the minimal test fails with assertions on and off; with the fix it passes, on and off.
• With the fix: TestThreadedForceMerge, TestIndexWriterForceMerge, TestForceMergeForever, TestIndexWriterMerging, TestConcurrentMergeScheduler, TestIndexWriterMergePolicy, TestLogMergePolicy, TestTieredMergePolicy and TestIndexWriter (with testRandomOperations): 211 tests, 7 skipped, 0 failures.
• Lint with the fix and the test in place: google-java-format flags only the test's long comment, not the fix; ecj and forbidden-APIs checks pass.
• Earlier, in the hunt: the hunt's own JUnit and 19 upstream forced-merge tests pass with the fix (standalone runner).
• Not covered: three or more overlapping calls, mixed blocking and non-blocking calls, overlaps with forceMergeDeletes, merge failures during an overlap, the full core suite.

Before the reset: if (maxNumSegmentsMergesPending()) maxNumSegments = Math.min(maxNumSegments, mergeMaxNumSegments); (the independent reproducer's root-cause probe).

Trade-offs

Smallest change, and while a blocking caller waits there is always such a merge; it also tightens a later call when only a non-blocking call's merges are still running; but between a call's reset and its maybeMerge no forced merge is pending yet, so a second call that resets in that window is not tightened. Checked only with the reproducer's own 3 tests through a shadowed class, not with gradle.

When no forced merge is left, a caller whose own segments are still above its n re-plans with its own n.

Trade-offs

Exact per-call semantics, but needs per-call lineage (the shared segmentsToMerge is cleared by every call) and changes the wait loop; not run.

State that overlapping forceMerge calls share one target, the most recent call's maxNumSegments, and that callers who need a specific result must serialize them.

Trade-offs

No code change and matches the LUCENE-1335 stance, but leaves a normal return that does not meet the @param; not run.

Method: TLA+ model checking (TLC) of IndexWriter.forceMerge(n, doWait=true) at the granularity of the writer's synchronized blocks (the study's "formal" tier: the tool's own counterexample gave the first signal).

• The model was set up with two callers that have different maxNumSegments, with no mechanism in mind.
• Invariant F1: a normal return leaves the segments present at the call merged into at most n segments. Configuration two_nofail_diff (two callers, n = 1 and n = 2, no merge failures) violated F1 for the n = 1 caller in a 13-state trace; two_nofail_same (n = 1 twice) was clean at 26,698 distinct states. A liveness check with n = 1 and n = 2 (every caller eventually returns) held at 916,793 states, so it is an early return, not a hang.
• The unit's first finder judged the trace not reportable, because the target is writer-wide by design; a second AI tool working the same unit wrote it up from the counterexample, replayed it on the stock jar as a JUnit, and the independent checks below reproduced it with their own tests.