Lucene 11.0 (main) turns incremental doc-values updates on by default, but neither its release notes nor the migration guide say so

Lucene 10.6.0 added incremental doc-values updates (GITHUB#16418): a doc-values update that only sets values is written as a small sparse overlay holding just the updated documents, layered over the column at read time, instead of a rewrite of the whole column. In 10.6 the feature is off unless the application calls IndexWriterConfig#setMaxDocValuesOverlays(n) (DEFAULT_MAX_DOC_VALUES_OVERLAYS = 0). On main and branch_11x, the future 11.0, it is on by default with up to 8 overlays per field (IndexWriterConfig.java:115), deliberately: the #16418 PR says "On main the feature is on by default".

The release notes do not record that difference. The only GITHUB#16418 entry in CHANGES.txt sits in the 10.6.0 section and says "Opt-in; enable with IndexWriterConfig#setMaxDocValuesOverlays(n)" (CHANGES.txt:501-504), correct for 10.6.0. The 11.0.0 "Changes in Runtime Behavior" list has only "The query cache is now disabled by default" (:341-343), and MIGRATE.md's 10.x-to-11.0 guide has no line on overlays, although it gives the query-cache flip a section with the code to undo it (MIGRATE.md:227-236). Main had an 11.0.0 entry saying "Enabled by default; disable with IndexWriterConfig#setMaxDocValuesOverlays(0)" until ea248114ac4 (2026-09-01) moved it to the 10.6.0 section and the 10.6.0 sync 7cecf795236 (2026-10-21) reworded it to the 10.6.0 text.

An application that upgrades from 10.6 to 11.0 with a default IndexWriterConfig and uses doc-values updates therefore moves to the overlay write path, with more live files per field, and nothing in the 11.0 notes tells it. Results stay correct, and in the measured example (1 of 1,000 documents) the update wrote 402 instead of 2,892 bytes. The fix is text only: an 11.0.0 CHANGES entry and a MIGRATE.md section with the opt-out. Upstream main (now 12.0.0-SNAPSHOT, with 11.0.0 as its second section) and branch_11x still lack it on 2026-11-01.

// IndexWriterConfig.java:110-115
/**
 * Default maximum number of sparse doc-values overlays a field keeps before they are folded into
 * one: {@code 8}. {@code 0} disables the feature (classic full-column rewrite); higher values
 * lower write amplification but keep more overlays (hence more files) live to merge at read time.
 */
public static final int DEFAULT_MAX_DOC_VALUES_OVERLAYS = 8;

// ReadersAndUpdates.java:363
final boolean sparseDelta = config.getMaxDocValuesOverlays() > 0 && anyRemoval == false;

• Every new config takes the default (LiveIndexWriterConfig.java:146). The setter's javadoc says "Enabled by default (see {@link #DEFAULT_MAX_DOC_VALUES_OVERLAYS})." and "Only takes effect when IndexWriter is first created." (IndexWriterConfig.java:342, :344); the setter is @lucene.experimental (:346), the update methods are not (IndexWriter.java:1962, :1996, :2037, :2070).
• The write path. A set-only update becomes a sparse delta generation (ReadersAndUpdates.java:363); once a field has 8 deltas the next write folds them into one (:368). The comment at :358-362 says single-valued sorted-numeric updates take the same path (GITHUB#16580, CHANGES.txt:523-527).
• 10.6.0 (tag 4ae946e5a9b): IndexWriterConfig.java:116 is = 0 and :343 says "Disabled by default"; its CHANGES.txt has the same "Opt-in" entry. 10.6.0 is consistent.

1. 75eddfaff22 (PR #16418, 2026-08-13): the feature lands on main, default 16, with an 11.0.0 entry ending "Enabled by default; disable with IndexWriterConfig#setMaxDocValuesOverlays(0)".
2. ea248114ac4 (2026-09-01, "Rename SegmentInfos.VERSION_11_0 to VERSION_10_6", pushed without a PR): the entry moves unchanged from 11.0.0 to 10.6.0, the day the opt-in 10.x backport (#16554) lands. The 11.0.0 section now says nothing.
3. 92d9f2c7faa (PR #16648, 2026-09-08): default 16 -> 8; its commit message says "The feature is not yet released, so no CHANGES entry is needed."
4. 7cecf795236 (2026-10-21, "Ensure 10.6.0 changelog in main is consistent with branch_10_6", pushed without a PR): the moved entry gets 10.6.0's "Opt-in" text. Correct for 10.6.0, but no "enabled by default" wording is left in main's notes.

• Who. Applications that use doc-values updates (updateNumericDocValue, updateBinaryDocValue, updateDocValues; single-valued sorted-numeric updates per GITHUB#16580) and upgrade to 11.0 with a default IndexWriterConfig. No opt-in or rare setting is needed; the update methods are public and not experimental.
• What changes. Set-only updates write sparse overlays instead of a full column (402 instead of 2,892 bytes of update files for 1 of 1,000 documents in the measured example), and up to 8 overlays per field stay live until a fold, so more files are live per field and reads merge them (IndexWriterConfig.java:110-114). #16648 lowered the default from 16 to 8 because "open file handles scale with segments times updated fields times the overlay budget", after a CI open-handle guard tripped (#16644).
• What does not change. Search results: the updated values read back correctly in every mode an independent reproducer tested, and nothing threw in those runs.
• Upgraders run into it. A downstream issue names this default as one of two Lucene 11 changes that affect a test failing against the Lucene 11 snapshot (elastic/elasticsearch#163787, 2026-10-28: "It also turns on doc-values overlays by default (DEFAULT_MAX_DOC_VALUES_OVERLAYS 0→8)"). It names the constant and does not cite Lucene's CHANGES or MIGRATE; the test still fails "despite the adjustments made", so the issue does not show the default to be the cause.
• Scale. Every such upgrader whose updates only set values (updates with removals still rewrite the whole column, ReadersAndUpdates.java:363), with no data-size threshold. Unreleased: no release ships the default without a note yet; 11.0.0 would be the first. Impact class minor: a documentation defect with no wrong results.

The hunt's JUnit (TestDocValuesOverlaysDefaultAsReleaseNotes) is not used as the repro: it asserts the 10.6.0 "Opt-in" note against main, which main's own javadoc contradicts, so it shows only that the default differs. On main 645d4fe14a20 it fails 2 of 2 with -ea and -da (new IndexWriterConfig().getMaxDocValuesOverlays() is 8; "with the default config an update of 1 of 1000 documents wrote 402 bytes of update files, with setMaxDocValuesOverlays(0) 2892 bytes"); on jars built locally from the 10.6.0 tag commit 4ae946e5a9b it passes (the default config and setMaxDocValuesOverlays(0) write the same number of bytes of update files). The step numbers come from that run.

No regression test is suggested: a test that greps CHANGES.txt would assert a project practice that no document promises.

The check is a term search: a note worded without its terms (say, only "incremental doc-values updates") would not be counted.

With the recommended fix applied to 645d4fe14a20, the same check prints:

# sh check_11_0_notes.sh $LUCENE   (scratch worktree of 645d4fe14a20 + fix-recommended-645d4fe.diff, 2026-11-01)
code default: DEFAULT_MAX_DOC_VALUES_OVERLAYS = 8
CHANGES.txt:345 [11.0.0] * GITHUB#16418, GITHUB#16648: Incremental doc-values updates, opt-in in 10.6, are now enabled by default:
CHANGES.txt:346 [11.0.0]   IndexWriterConfig#DEFAULT_MAX_DOC_VALUES_OVERLAYS is 8. Pass IndexWriterConfig#setMaxDocValuesOverlays(0)
CHANGES.txt:505 [10.6.0] * GITHUB#16418: Incremental doc-values updates. A set-only doc-values update (NUMERIC or BINARY) is written as a sparse
CHANGES.txt:507 [10.6.0]   rewriting the whole column, turning per-update write amplification from O(column) into O(updated docs). Opt-in;
CHANGES.txt:508 [10.6.0]   enable with IndexWriterConfig#setMaxDocValuesOverlays(n). (Jim Ferenczi)
lines on it in the 11.0.0 section of CHANGES.txt: 2
lines on it in MIGRATE.md: 3
OK: the 11.0 notes record the feature
exit=0

# sh check_11_0_notes.sh $LUCENE   (main 645d4fe14a20, 2026-11-01)
code default: DEFAULT_MAX_DOC_VALUES_OVERLAYS = 8
CHANGES.txt:501 [10.6.0] * GITHUB#16418: Incremental doc-values updates. A set-only doc-values update (NUMERIC or BINARY) is written as a sparse
CHANGES.txt:503 [10.6.0]   rewriting the whole column, turning per-update write amplification from O(column) into O(updated docs). Opt-in;
CHANGES.txt:504 [10.6.0]   enable with IndexWriterConfig#setMaxDocValuesOverlays(n). (Jim Ferenczi)
lines on it in the 11.0.0 section of CHANGES.txt: 0
lines on it in MIGRATE.md: 0
MISSING: no 11.0 note says the default changed
exit=1

• The project's own practice. Default changes are listed under "Changes in Runtime Behavior": in this 11.0.0 section "GITHUB#14187: The query cache is now disabled by default." (CHANGES.txt:343), and earlier "GITHUB#15040: The default ReadAdvice has been changed from RANDOM to NORMAL" (10.3.0, :1739) and "GITHUB13293: Auto I/O throttling is now disabled by default on ConcurrentMergeScheduler" (10.0.0, :2403). MIGRATE.md's 11.0 guide has sections for other default changes, among them the query cache (:227-236) and the filtered-HNSW threshold (:238-244).
• The record existed and was lost as a side effect. The 11.0.0 entry said "Enabled by default" until ea248114ac4, a commit about renaming a SegmentInfos constant, moved it to 10.6.0; the 10.6.0 sync then reworded it, correctly for 10.6.0. No PR, issue or dev@ thread found discusses leaving 11.0 without a note.
• The one "no CHANGES entry" decision was about something else. #16648's commit message skips an entry for the 16 -> 8 step because "the feature is not yet released". It was written while main's entry still said "Enabled by default" (by then under 10.6.0), and its premise ended when 10.6.0 shipped the feature opt-in (tag 2026-10-29).
• Upstream merges this kind of fix. #16912, "Move CHANGES entry for GITHUB#16794 from 10.6.0 to 11.0.0", was merged on 2026-10-26.
• What is not claimed. That the code is wrong (default-on is deliberate and documented in the javadoc), that the 10.6.0 entry is wrong, that results change, or anything about soft-delete updates or BINARY updates beyond the CHANGES text (not tested separately).

Strongest case for "intended / noise". Default-on is deliberate and documented where the API is documented: IndexWriterConfig.java:342 says "Enabled by default", the #16418 and #16554 PR texts state the main/10.x difference, and TestIncrementalDocValuesUpdates.testDefaultBudgetHeavyStackingAndFolding "Exercises the shipping default overlay budget". The setter is @lucene.experimental. CHANGES entries are optional (CONTRIBUTING.md:107: "You may want to add a CHANGES entry"), the release manager distills the release notes from CHANGES.txt before a release (dev-tools/scripts/releaseWizard.yaml:606-610), and 11.0 is unreleased. The 10.6.0 entry is correct for 10.6.0, and results never change. The cross-model refute lens argued this and rated it not real; an intended-behaviour verifier built the same case.

Why it fails. It defeats the framing "the code is wrong", which nobody claims, not the narrow claim. Nothing shows that the 11.0 record was meant to go (the history shows the edits, not their intent, and no discussion found decides it): the javadoc of an experimental setter that an upgrader keeping the defaults has no reason to call is not where upgraders look; the same section and MIGRATE.md record the analogous flips; the record existed and disappeared through two changelog edits; and the release manager's draft starts from CHANGES.txt, so an explicit entry is the direct way into it. The refute lens conceded that neither the 11.0 CHANGES nor MIGRATE records the change. The adjudication ruled that these points go to severity (minor), not to reality.

Text-only fixes. Upstream main now has a 12.0.0 section on top, so the entry goes into main's 11.0.0 section and is backported to branch_11x; the anchors are the GITHUB#14187 entry and the query-cache MIGRATE section.

An entry under 11.0.0 "Changes in Runtime Behavior", next to GITHUB#14187, and a MIGRATE.md section after "Query caching is now disabled by default" that says what changed and how to keep the 10.x behaviour.

--- a/lucene/CHANGES.txt
+++ b/lucene/CHANGES.txt
@@ -342,6 +342,10 @@ Changes in Runtime Behavior
 ---------------------
 * GITHUB#14187: The query cache is now disabled by default. (Adrien Grand)
 
+* GITHUB#16418, GITHUB#16648: Incremental doc-values updates, opt-in in 10.6, are now enabled by default:
+  IndexWriterConfig#DEFAULT_MAX_DOC_VALUES_OVERLAYS is 8. Pass IndexWriterConfig#setMaxDocValuesOverlays(0)
+  to keep rewriting the whole column on every doc-values update. (Jim Ferenczi)
+
 Build
 ---------------------
 * GITHUB#15327: New low-level build options to detect abuse of LuceneTestCase.random():
--- a/lucene/MIGRATE.md
+++ b/lucene/MIGRATE.md
@@ -235,6 +235,21 @@ long maxRamBytesUsed = 50 * 1024 * 1024; // 50MB
 IndexSearcher.setDefaultQueryCache(new LRUQueryCache(maxCachedQueries, maxRamBytesUsed));
 ```
 
+### Incremental doc-values updates are enabled by default (GITHUB#16418, GITHUB#16648)
+
+Incremental doc-values updates, opt-in in Lucene 10.6, are now enabled by
+default. A doc-values update that only sets values (no removals) is written as
+a sparse overlay holding just the updated documents instead of a rewrite of
+the whole column, and a field keeps up to
+`IndexWriterConfig.DEFAULT_MAX_DOC_VALUES_OVERLAYS` (8) overlays before they
+are folded into one, so more files per field can stay live. To keep the
+full-column rewrite of 10.x, disable the feature when you create the writer:
+
+```java
+IndexWriterConfig config = new IndexWriterConfig(analyzer);
+config.setMaxDocValuesOverlays(0);
+```
+
 ### Filtered HNSW search default threshold (GITHUB#14160)
 
 Filter-optimized HNSW search now runs when 60% or fewer vectors pass the

Trade-offs

No behaviour change. The CHANGES line reaches readers of the release notes; the MIGRATE section reaches people planning the upgrade and gives the opt-out. The MIGRATE wording follows the javadoc (IndexWriterConfig.java:110-114, :335-342) and claims nothing about file counts beyond "more files per field can stay live".

What the checks validated

Scratch worktree of 645d4fe14a20, OpenJDK 25.0.3, 2026-11-01:

• the notes check prints [11.0.0] lines, "11.0.0 section: 2", "MIGRATE.md: 3", OK, exit 0;
• ./gradlew :lucene:documentation:changesToHtml :lucene:documentation:markdownToHtml succeeds: the 11.0.0 "Changes in Runtime Behavior" list renders with 2 entries and MIGRATE.html has the new section;
• ./gradlew :lucene:core:test --tests TestIndexWriterConfig --tests TestIncrementalDocValuesUpdates passes: 24 tests, 0 failures (the diff changes only text, so this only confirms the tree still builds).

The first hunk of the recommended diff (the hunt's original fix).

Trade-offs

Smallest change, but MIGRATE.md is the upgrade guide, and the analogous query-cache flip is in both files.

Set DEFAULT_MAX_DOC_VALUES_OVERLAYS = 0 on main.

Trade-offs

Not suggested: it reverses a deliberate decision (#16418, #16648) with a code change to fix a missing sentence.

The status quo.

Trade-offs

The release manager drafts the release notes from CHANGES.txt (releaseWizard.yaml:606-610), so an explicit CHANGES entry is the direct way into them, and branch_11x is already cut.

Method: non-formal. The oracle is the release notes plus the 10.6 behaviour, checked by reading and a differential test; no model, proof or bounded-exhaustive check took part.

• The unit's optional main-only part started from a note that main defaults to 8 overlays with no 11.0 CHANGES or MIGRATE entry.
• The finder compared CHANGES.txt, MIGRATE.md and IndexWriterConfig on main and on the 10.6.0 release-candidate jars, and wrote a public-API JUnit: the same one-document update with a plain new IndexWriterConfig() and with setMaxDocValuesOverlays(0).
• It traced the provenance of the entry in the full git history; a verifier corrected the introducing commit to ea248114ac4.
• Because a missing note cannot be unit-tested, a read-only check replaced the JUnit as the minimal repro (this doc's repro).