TemporalMergePolicy: forceMergeDeletes() skips a segment that is the only one over the delete threshold in its time window

TemporalMergePolicy (opt-in, @lucene.experimental, first released in 10.5.0) groups segments into time windows and never merges across windows. For IndexWriter.forceMergeDeletes() its javadoc says it merges segments with more than forceMergeDeletesPctAllowed (default 10%) deleted documents. But findForcedDeletesMerges skips every window with fewer than two such segments, and its batching helper drops the one left over after full batches of maxThreshold (default 8). So a segment that is the only eligible one in its window keeps its deleted documents: a one-segment index with 1 of 2 documents deleted keeps maxDoc 2, repeated calls schedule nothing, and forceMerge(1), which has the same guard, does not rewrite it either. Searches stay correct; the cost is disk space and search work. Affected: 10.5.0, 10.5.1, 10.5.2, 10.6.0 and main.

• Time window: TemporalMergePolicy assigns each segment to a window by the largest value of its LongPoint timestamp field. Recent windows are small (1 hour by default) and older ones grow by a factor of minThreshold (4) up to 365 days; disableExponentialBuckets() makes them all the same size (baseTimeSeconds, without the 365-day cap). Segments of different windows are never merged, "even during forceMerge(int)" (class javadoc, TemporalMergePolicy.java:87-89).
• Eligible segment: for forceMergeDeletes(), a segment that is not already merging and whose share of deleted documents is above forceMergeDeletesPctAllowed (default 10.0, :111, javadoc :353-359).

1. IndexWriter.forceMergeDeletes() first flushes and applies the deletes (IndexWriter.java:2348). That flush also runs a natural merge pass (findMerges, :4378-4379, :2508), which for this policy needs 4 segments in a window and finds nothing here. Then it calls findForcedDeletesMerges once (:2359). If the policy returns nothing, no forced merge runs and the call returns without waiting (:2369, :2405).
2. TemporalMergePolicy.findForcedDeletesMerges (:527-598) computes pctDeletes = (100.0d * delCount) / maxDoc (:549-554), keeps the segments above the threshold (:555) and groups them by window (:568).
3. Per window: if (bucketSegments.size() < 2) { continue; } (:573-575). A window with one eligible segment is dropped, whatever else it holds.
4. Windows with two or more go to buildSequentialMerges (:951-978). It returns null for fewer than two candidates (:952-954), emits batches of exactly maxThreshold (default 8, :108), and emits the last batch only if (batch.size() > 1) (:970). So 9 eligible segments in a window leave the 9th with its deletes.
5. Natural merges never look at deletes and need at least minThreshold (4) segments in a window (:833), so the deletes stay until enough new segments land in that window. With the default growing windows an aging segment can later share a bigger window and be merged then. That may never happen with fixed windows (all baseTimeSeconds wide, :796-797), for data old enough to sit in the capped 365-day windows (:803-809), or in a single window or a single-segment index.
6. The forceMerge twin. findForcedMerges (:507-509) and buildForcedMerges (:925-927) have the same < 2 guard, so forceMerge(1) does not rewrite a lone segment with deletes either. forceMerge still keeps its promise (at most N segments, never across windows); only the deletes stay.

Default settings, one segment of 2 documents, 1 deleted (50%): forceMergeDeletes() schedules 0 merges and maxDoc stays 2; three calls in a row, then forceMerge(1): still maxDoc 2, hasDeletions() true. The verifiers' wider layouts on the same commit: two windows with one 50%-deleted segment each keep both ([10/5][10/5]); one eligible segment among three in a window keeps its deletes ([10/0][10/5][10/0]); 9 eligible segments in one window leave _6 c10/5 beside the merged _9 c40; forceMerge(1) on one segment with 5 of 10 deleted leaves _0 c10/5. Two eligible segments in one window are merged correctly. LogMergePolicy rewrites the minimal repro's segment in the same test (1 merge, maxDoc 1). TieredMergePolicy is not a clean control for that exact test: the flush inside forceMergeDeletes() first starts a natural merge of the segment, so the call reports 0 merges and returns with maxDoc 2 (1 once the background merge finishes). Its forced-deletes path does return the one-segment merge when it alone decides, and the verifiers' TieredMergePolicy controls (their own layouts, a serial merge scheduler, the writer closed before checking) expunge every layout.

• Who is hit. Users of TemporalMergePolicy (@lucene.experimental, TemporalMergePolicy.java:94; first released in 10.5.0) who call forceMergeDeletes() to reclaim deletes. Default settings and the public API are enough, and the outcome is deterministic.
• Typical trigger states. A single-segment index; any index after forceMerge(1) (one segment per window); an old window whose segments were already merged into one; the odd segment after full batches of 8.
• What it costs. Hits stay correct. The deleted documents stay in the segment files, still count in maxDoc, docFreq and other term statistics, keep their terms in the terms dictionary (a spell checker still suggested a term found only in a deleted document), and query paths that visit them must skip them. They never come back as hits (checked before and after the call). Nothing is corrupted or lost.
• No fix from this policy's own calls. Repeating the call schedules nothing again, and forceMerge(1) skips the segment too. Natural merges ignore deletes. A workaround, partly run: on a one-segment layout with a serial merge scheduler, switching the live config to a TieredMergePolicy with setForceMergeDeletesPctAllowed(0) and calling forceMergeDeletes(true) removed the deletes. Then restore the TemporalMergePolicy and commit (not run). With the default concurrent scheduler check hasDeletions() only after the merges finish; with several windows this can mix windows (by reading).
• Scale. Any index size; the cost is the deleted documents of each skipped segment. Fully deleted segments are dropped by IndexWriter anyway.
• Tested with hard deletes only. Every run used deleteDocuments; soft deletes were not tested.
• How likely. Plausible: the trigger states are normal for this policy, but it is new and experimental, no downstream user is known, and forceMergeDeletes is an explicit call that IndexWriter's javadoc says "rarely is it warranted".

Package org.apache.lucene.index, like the policy's own tests; one method; default IndexWriterConfig with only the merge policy set; ByteBuffersDirectory. The timestamp is a fixed value in seconds; any value gives the same result because the segment is alone in its window. Nothing else is randomized that matters: two seeds give the same failure with assertions on and off.

With the recommended fix the same file is the regression test: it passes, and :lucene:core:check -x test (google-java-format, ecj lint, forbidden APIs) passes with it in the tree. The wider layouts (one eligible among several, one per window, the leftover after a full batch, the forceMerge(1) twin) are covered by the verifiers' runs, not by more tests.

cd $LUCENE   # apache/lucene at 645d4fe14a20 (or any commit with the same TemporalMergePolicy.java)
cp $REPRO/TestTemporalMergePolicyForceMergeDeletes.java lucene/core/src/test/org/apache/lucene/index/
./gradlew :lucene:core:test --tests org.apache.lucene.index.TestTemporalMergePolicyForceMergeDeletes -Ptests.seed=DEADBEEF
# assertions off fails the same way: add -Ptests.asserts=false

# gradle, main 645d4fe14a20, seed DEADBEEF, run with TestTemporalMergePolicy*
TestTemporalMergePolicyForceMergeDeletes > testForceMergeDeletesSingleSegment FAILED
    java.lang.AssertionError: maxDoc after forceMergeDeletes expected:<1> but was:<2>
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:4DFDDAB49229B500]:0)
        at org.junit.Assert.assertEquals(Assert.java:647)
        at org.apache.lucene.index.TestTemporalMergePolicyForceMergeDeletes.testForceMergeDeletesSingleSegment(TestTemporalMergePolicyForceMergeDeletes.java:43)
:lucene:core:test (FAILURE): 12 tests, 1  failure     # TestTemporalMergePolicy 7/7 and ...AcrossBuckets 4/4 pass

# with the recommended fix: 12 tests, 0 failures (assertions on and off)

# wider layouts (the verifiers' suites, same commit, seed DEADBEEF)
perSegment(maxDoc/del)=[10/5][10/5] expected:<0> but was:<10>          # one 50%-deleted segment in each of two windows
perSegment(maxDoc/del)=[10/0][10/5][10/0] expected:<0> but was:<5>     # one eligible among three in a window
deletes left after forceMergeDeletes (9 segs, maxThreshold 8): segments_a: _9(11.0.0):c40 _6(11.0.0):c10/5
deletes left after forceMerge(1): segments_2: _0(11.0.0):c10/5

• The base contract. MergePolicy.findForcedDeletesMerges: "Determine what set of merge operations is necessary in order to expunge all deletes from the index" (MergePolicy.java:654-656). TemporalMergePolicy overrides it with no javadoc that narrows it (TemporalMergePolicy.java:527-528).
• The policy's own rule is the percentage. setForceMergeDeletesPctAllowed: "When IndexWriter#forceMergeDeletes() is called, only merge segments whose delete percentage exceeds this threshold ... Default is 10.0 (merge segments with more than 10% deleted documents)" (:353-359).
• The documented exemptions do not apply. The class NOTEs exempt merges across windows (:87-89) and segments older than maxAgeSeconds (:91-92; off by default, Long.MAX_VALUE, :106-107).
• A one-segment merge is legal. OneMerge only needs a non-empty list (MergePolicy.java:226-229).
• The other core policies do it. TestTieredMergePolicy has "Test singleton merge for expungeDeletes" (:407-415, LUCENE-7976); LogMergePolicy.findForcedDeletesMerges emits merges of any length, one included (LogMergePolicy.java:438, 450, 461). In the review of #15620, which added the policy, benwtrent noted that the PR summary did not say how "deletes and updates" are handled; the author answered that IndexWriter.forceMergeDeletes() "works as expected, but there are a couple of caveats"; benwtrent then said they would expect merging "with only expunging deletes" to work "as normal". The caveats in the PR description cover only natural merges and the same-window rule.
• No way out under this policy. If the skip were deliberate, neither forceMergeDeletes() nor forceMerge(1) would reclaim a partially deleted lone segment while TemporalMergePolicy is active; only switching to another policy does.

The strongest case for "intended": the policy's author wrote TestTemporalMergePolicy.testForcedDeletesKeepWindowsDisjoint (:200-235) with the code, already in the PR's first commit. It gives segB1 12% deletes against a 5% threshold, alone in its window, and asserts exactly one merge, of segA1 and segA2 only (:230-235). The guard is written five times; the javadoc's "only merge segments whose delete percentage exceeds" can be read as a necessary, not sufficient, condition; IndexWriter.forceMergeDeletes says the actual merges "are determined by the MergePolicy" (IndexWriter.java:2409-2410); the PR description says forced deletes "merges those within the same time window", which can be read as two or more; NoMergePolicy shows a policy may return no forced-deletes merge at all; and for append-heavy time-series data skipping a whole-segment rewrite may be a deliberate I/O trade-off (the author made that call for natural merges: "for us it wasn't worth the extra I/O"). Two of the cross-model reviews (refute and root-cause lenses, reading only) rated this strong, and so did the cross-model fact check of this doc; the intended-lens review and the intended-behaviour check rated it moderate.

Why it fails: the test's name and its sibling testForcedMergeKeepsWindowsDisjoint (:155-198) show what it checks, that no merge mixes windows; rewriting segB1 alone keeps that, and the updated assertion of the recommended fix still checks it. Nothing else documents the skip: no javadoc, CHANGES or MIGRATE text, no review comment in #15620. The javadoc's "Default is 10.0 (merge segments with more than 10% deleted documents)" says what gets merged, and the base contract asks for all deletes. The PR description's sentence describes the same-window rule and names no minimum of two; NoMergePolicy exists to merge nothing, while this policy implements forced deletes; and the threshold already gates the I/O, as in TieredMergePolicy. So the case is "probably not intended" (medium confidence), not settled. If maintainers still call it intended, the fallback is a javadoc change. For the forceMerge(1) twin the intended reading is stronger: a one-segment window already meets its segment-count goal.

Three options. A was run with the repro, the policy's test classes (assertions on and off) and :lucene:core:check -x test; A + B also with TestTieredMergePolicy and TestLogMergePolicy; all on a scratch worktree of 645d4fe14a20, removed afterwards.

Three conditions in TemporalMergePolicy.java (:573, :952, :970) and the pinned assertion in TestTemporalMergePolicy.testForcedDeletesKeepWindowsDisjoint, which now expects two merges, {segA1, segA2} and {segB1}, so it still checks that no merge mixes windows.

// findForcedDeletesMerges, per window
-      if (bucketSegments.size() < 2) {
+      if (bucketSegments.isEmpty()) {
// buildSequentialMerges
-    if (candidates.size() < 2) {
+    if (candidates.isEmpty()) {
-    if (batch.size() > 1) {
+    // a single leftover segment is rewritten on its own to expunge its deletes
+    if (batch.isEmpty() == false) {

Trade-offs

• Each lone eligible segment is rewritten on its own, at the cost of rewriting its size; that is what the caller asked for, and the threshold still gates it, as in TieredMergePolicy. It happens once: a second call finds no deletes and runs 0 merges.
• The guard at :573 can never fire (windows are built from candidates), so it could also be removed.
• findForcedDeletesMerges does not skip the old-data bucket (segments older than maxAgeSeconds, off by default), unlike findMerges and findForcedMerges; with the fix a lone old segment over the threshold is rewritten too. Whether forced deletes should respect maxAgeSeconds is a separate question (by reading, not run).
• It changes one upstream assertion, and it leaves forceMerge as it is (option B).

What the checks validated

• Stock: the repro fails (12 tests, 1 failure). With A: the repro, TestTemporalMergePolicy (7) and TestTemporalMergePolicyAcrossBuckets (4) pass with assertions on and off (12 tests, 0 failures); :lucene:core:check -x test passes with the repro test in the tree.
• Without the assertion change the pinned test fails, expected:<1> but was:<2> (run on ea57360912ed, same file).
• The verifiers' suites: the independent reproduction 8 of 8 pass (stock 4 of 8 fail); a separate test 3 of 4 (its forceMerge(1) test still fails, as expected). A retry probe: the first call runs 1 merge (maxDoc 1), the next two run 0.

In findForcedMerges (:507), a window with one segment gets a one-segment merge when mergeContext.numDeletesToMerge(segment) > 0; +8 lines. A lone segment without deletes is left alone, and the old-data bucket is still skipped (:503). This matches TieredMergePolicy, which bails out of a one-segment forceMerge(1) only when there are no deletes (TieredMergePolicy.java:834-849, from LUCENE-7976, and the merge-down-to-one case at :865-874, which LUCENE-8688 kept so that forceMerge of a single segment with deletes is not a no-op). An optional consistency change: a one-segment window already meets forceMerge(1)'s segment-count goal.

      if (bucketSegments.size() < 2) {
        // a window's lone segment is rewritten on its own only if it has deletes
        SegmentCommitInfo lone = bucketSegments.getFirst();
        if (mergeContext != null && mergeContext.numDeletesToMerge(lone) > 0) {
          if (spec == null) {
            spec = new MergeSpecification();
          }
          spec.add(new OneMerge(List.of(lone)));
        }
        continue;
      }

Trade-offs

Widens the change to forceMerge. forceMerge(n) with n > 1 can still leave deletes in a window that already has at most its share of segments; that is within forceMerge's contract and not touched (by reading).

What the checks validated

With A + B: TestTemporalMergePolicy*, TestTieredMergePolicy and TestLogMergePolicy pass (42 tests, 2 skipped, 0 failures); :lucene:core:check -x test passes; a separate test 4 of 4 (incl. forceMerge(1)), the independent reproduction 8 of 8 and the follow-up's residue probe 5 of 5 pass.

Say in setForceMergeDeletesPctAllowed and the class NOTEs that a window with a single eligible segment is not merged.

Trade-offs

Closes the contract gap on paper but leaves no call under this policy that reclaims those deletes; the repro still fails. A fallback only if maintainers want today's behaviour.

Method: non-formal (code reading, then executed tests).

• Finding. A hunt over Lucene's merge policies had a bounded-exhaustive check over synthetic segments; TemporalMergePolicy reads point values from each segment, so it was outside that check. Reading its forced-deletes path showed the < 2 guards.
• Confirmation. A JUnit through IndexWriter on the stock build (one segment with 5 of 10 deleted; two windows with one 50%-deleted segment each) failed with assertions on and off and on 10.5.0 to 10.5.2, and passed with the candidate fix.
• Then cross-model reviews (OpenAI GPT-5.6, reading only), an independent reproducer with its own public-API suite and controls, a root-cause review, an intended-behaviour devil's advocate, an adjudication, a history and prior-report follow-up, a security screen, a main refresh, a repro-minimization step and this doc's gradle and fix runs.