DictionaryCompoundWordTokenFilterFactory on main warns that onlyLongestMatch "has been removed" when the key is not set

On main, DictionaryCompoundWordTokenFilterFactory (SPI name dictionaryCompoundWord) still reads the removed onlyLongestMatch key, now only to decide whether to log: boolean onlyLongestMatch = getBoolean(args, "onlyLongestMatch", true); if (onlyLongestMatch) LOG.warning("onlyLongestMatch has been removed and replaced by onlyLongestMatchIgnoreSubwords");. Because getBoolean returns the default true for a missing key, every construction that does not mention the key logs the warning: the plain dictionary=... configuration that Lucene's own factory test uses, and configurations already migrated to onlyLongestMatchIgnoreSubwords. A configuration that names the now-ignored key with false gets no warning, and the class javadoc example still shows onlyLongestMatch="true". Only the log is affected: create() passes only onlyLongestMatchIgnoreSubwords to the filter. The lines came with #14356 (ebc63504bd6, merged 2025-05-10, an 11.0 API removal) and are in no release; 9.12.3 through 10.6.0 log nothing. Warning when the key is present, whatever its value, fixes it.

• AbstractAnalysisFactory.getBoolean(args, name, defaultVal): removes the setting from the map and returns its value, or defaultVal when it is absent: String s = args.remove(name); return s == null ? defaultVal : Boolean.parseBoolean(s); (lucene/core/src/java/org/apache/lucene/analysis/AbstractAnalysisFactory.java:213-216 at 645d4fe14a20).
• onlyLongestMatchIgnoreSubwords: the replacement setting, read with default true (DictionaryCompoundWordTokenFilterFactory.java:74). The filter javadoc says it supersedes onlyLongestMatch (DictionaryCompoundWordTokenFilter.java:54-56).

1. The constructor reads the removed key and warns on its value (DictionaryCompoundWordTokenFilterFactory.java:69-72):

boolean onlyLongestMatch = getBoolean(args, "onlyLongestMatch", true);
if (onlyLongestMatch)
  LOG.warning(
      "onlyLongestMatch has been removed and replaced by onlyLongestMatchIgnoreSubwords");

1. A missing key and "true" (in any letter case) give true. "false" gives false, and so does any other present value such as "yes" or "", because Boolean.parseBoolean returns true only for "true" (executed: a six-value probe on main). The default true goes back to the first factory version (2008), when the key still had a meaning.
2. The value is used for nothing else: create() passes only the dictionary, the size limits and onlyLongestMatchIgnoreSubwords to the filter (:97-103), whose 6-argument constructor hands false to the base class (DictionaryCompoundWordTokenFilter.java:58-66).
3. So the warning fires for every configuration that never set the key, and is silent for the one that set it to false. The class javadoc example still writes onlyLongestMatch="true" (:35-36), so following it also gives the warning.

Dictionary {donau, dampf, dampfschiff, schiff}, word donaudampfschiff, 7 configurations through the SPI name on main 645d4fe14a20:

The tokens depend only on onlyLongestMatchIgnoreSubwords, never on the warning. 10.5.2 and 10.6.0 log nothing in any configuration. 10.1.0 logs nothing either, and rejects the four configurations that set onlyLongestMatchIgnoreSubwords (added in 10.2.0) with "Unknown parameters".

• Who is hit. Applications on main (11.0.0-SNAPSHOT) that build the dictionaryCompoundWord factory: its public Map constructor, TokenFilterFactory.forName, CustomAnalyzer, or an analyzer definition loaded by SPI name. Solr schemas use this factory (solr.DictionaryCompoundWordTokenFilterFactory). In the Elasticsearch and OpenSearch code inspected on 2026-10-26 (a read-only code search), the filter is built directly, not through this factory; other versions, plugins and custom applications were not surveyed. No default analysis chain builds it.
• What it takes. Nothing unusual: the default configuration, exactly what Lucene's own TestDictionaryCompoundWordTokenFilterFactory.testDecompounding builds, or the replacement setting the warning itself recommends.
• What happens. One constant WARNING per factory construction, so once per analyzer build, never per document or query. Within the factory settings, a migrated configuration can silence it only by adding the removed key with a value other than "true" (ignoring case), conventionally false (otherwise only the logging configuration hides it); a configuration that still sets onlyLongestMatch="false", an ignored key, is not told.
• Scale. Minor: log diagnostics only, on unreleased code. It would ship in 11.0.0 unless fixed.

The dictionary file is read only in inform(), so d.txt need not exist and no resource loader is needed. The map is a mutable HashMap because the factory removes the settings it consumes. Pass -Ptests.asserts explicitly: on main, gradle picks assertions on or off from the seed, and seed DEADBEEF picks off. Seeds DEADBEEF, 5EED1234, C0FFEE01 and 1234ABCD all fail the same way. On 10.5.2 and 10.6.0 the test passes.

As the regression test: the same test, for example as a new method in TestDictionaryCompoundWordTokenFilterFactory. It covers the absent-key half only; that the fix still warns once when onlyLongestMatch is set (true or false) is shown by the executed matrix with the fix, not by this test.

cp TestDictionaryCompoundFactoryRemovedKeyWarning.java $LUCENE/lucene/analysis/common/src/test/org/apache/lucene/analysis/compound/
cd $LUCENE && ./gradlew :lucene:analysis:common:test --tests org.apache.lucene.analysis.compound.TestDictionaryCompoundFactoryRemovedKeyWarning -Ptests.seed=DEADBEEF -Ptests.asserts=true
# assertions off: -Ptests.asserts=false (same result)

# gradle, main 645d4fe14a20, seed DEADBEEF (the same with assertions on and off)
TestDictionaryCompoundFactoryRemovedKeyWarning > testNoWarningWhenRemovedKeyIsAbsent FAILED
    java.lang.AssertionError: expected:<[]> but was:<[onlyLongestMatch has been removed and replaced by onlyLongestMatchIgnoreSubwords]>
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:B2ABB632A32114A6]:0)
        ...
        at org.apache.lucene.analysis.compound.TestDictionaryCompoundFactoryRemovedKeyWarning.testNoWarningWhenRemovedKeyIsAbsent(TestDictionaryCompoundFactoryRemovedKeyWarning.java:56)

# the factory's own log line in the test output
  2> ... org.apache.lucene.analysis.compound.DictionaryCompoundWordTokenFilterFactory <init>
  2> WARNING: onlyLongestMatch has been removed and replaced by onlyLongestMatchIgnoreSubwords

• The message. "onlyLongestMatch has been removed and replaced by onlyLongestMatchIgnoreSubwords" is a statement about a key the configuration used. Logging it because a key is missing contradicts its own text.
• The project's stated mechanism for deprecated factory keys. On #14311, the PR that introduced the replacement setting, Uwe Schindler (uschindler) wrote on 2025-03-05: "We have no mechanism for deprecating those map keys of the factory. We can only log a warning, but have no standard way for this yet." On #15900 (2026-04-03) Uwe Schindler sketched the same pattern for another factory: "if you set prefixLength it uses the old buggy behaviour and logs a warning that it is deprecated". The first names the mechanism without saying when it should fire; the second ties the warning to the user setting the key. Both are context, not a rule for this factory.
• The author's own question. The only review comment of #14356's author, renatoh, sits on the if (onlyLongestMatch) hunk: "@rmuir Does this check make sense?" (2025-03-14). Nobody answered; the PR merged on 2025-05-10.
• No documentation or test sets it. No javadoc, CHANGES.txt entry, MIGRATE.md entry or test says when the factory logs. MIGRATE.md:648-668 (GITHUB#14356) covers only the removed filter constructor. TestDictionaryCompoundWordTokenFilterFactory has only testDecompounding and testBogusArguments.
• Lucene's canonical use is warned. testDecompounding builds the factory with only dictionary, as the minimal test does, so Lucene's own factory test takes the warning path.

The strongest case for "intended" (a read-only reviewer from another model family, OpenAI GPT-5.6, rated it moderate): the check flags everyone whose legacy onlyLongestMatch is effectively true, including by default, because that is the behaviour that went away, and spares an explicit false, because that configuration lost nothing.

Why it does not hold up (executed): with onlyLongestMatchIgnoreSubwords at its default true, onlyLongestMatch already had no effect on 10.5.2, because the filter ORs the two flags. So the default and the migrated configurations give the same tokens on 10.5.2 and main and never depended on the key, yet they are warned. Against 10.1.0, where onlyLongestMatch=false did change tokens, main does not warn that configuration. On neither upgrade path does the value-based condition pick the configurations whose output changed, the check ignores onlyLongestMatchIgnoreSubwords read on the next line, and nothing records such a policy as a decision. This weakens the intended reading but does not rule it out: the cross-model fact check of the brief (OpenAI GPT-5.6, read-only) again rated it moderate, a plausible migration-warning policy with an inconsistent condition. The maintainers decide.

One option was run through gradle in a scratch worktree of 645d4fe14a20 and is recommended, with a documentation add-on; the alternative was not run.

In the constructor (DictionaryCompoundWordTokenFilterFactory.java:69-72), replace the value check with a presence check that still consumes the key, so old configurations keep loading without "Unknown parameters". In the class javadoc (:36), replace onlyLongestMatch="true" with onlyLongestMatchIgnoreSubwords="true". The snippet is an excerpt of the full diff with context lines trimmed; the full diff applies with git apply at 645d4fe14a20. It is an equivalent one-call form of the adjudication's fix, which tests args.containsKey("onlyLongestMatch") and then calls args.remove(...).

@@ -33,7 +33,8 @@
  *     <filter class="solr.DictionaryCompoundWordTokenFilterFactory" dictionary="dictionary.txt"
- *         minWordSize="5" minSubwordSize="2" maxSubwordSize="15" onlyLongestMatch="true"/>
+ *         minWordSize="5" minSubwordSize="2" maxSubwordSize="15"
+ *         onlyLongestMatchIgnoreSubwords="true"/>
@@ -66,10 +67,11 @@
-    boolean onlyLongestMatch = getBoolean(args, "onlyLongestMatch", true);
-    if (onlyLongestMatch)
+    // removed key: still accepted, so old configurations load, but ignored; warn only if set
+    if (args.remove("onlyLongestMatch") != null) {
       LOG.warning(
           "onlyLongestMatch has been removed and replaced by onlyLongestMatchIgnoreSubwords");
+    }

Trade-offs

• Effect, executed on the 7-configuration matrix: the warning is logged for exactly the 4 configurations that set onlyLongestMatch (true or false) and for none of the 3 that do not; no token changes anywhere.
• onlyLongestMatchIgnoreSubwords="false" without the old key loses the warning it gets today by accident. Its tokens did change from 10.5.2 to main (dampf is now emitted for donaudampfschiff), because the 10.5.2 factory passed the default onlyLongestMatch=true through the constructor that #14356 removed. That change follows from the intended removal and is not this bug, but MIGRATE.md does not mention it: pair this fix with the MIGRATE line below.
• Diagnostics for unusual values (executed, six-value probe on main and with this change): today a present value other than "true", such as "yes" or "", is consumed with no message; with the fix it warns like any other use of the key. Probably wanted, but a behaviour change to mention.

What the checks validated

• Gradle in a scratch worktree of 645d4fe14a20 (2026-11-01): stock, the minimal test fails with assertions on and off; with the fix it passes both ways.
• With the fix: the compound package (TestCompoundWordTokenFilter, TestDictionaryCompoundWordTokenFilterFactory, TestHyphenationCompoundWordTokenFilterFactory, TestPatternParser) the repro and the nightly TestFactories (every factory with random arguments): 6 classes, 27 tests, 0 failures with assertions on; :lucene:analysis:common:check -x test passes (google-java-format, ecjLint, forbiddenApis).
• The same change shadowed into the stock main jars: the minimal test passes with -ea and -da, and the matrix shows the effect above.

Under the existing GITHUB#14356 entry, note that the factory setting onlyLongestMatch is ignored (a warning is logged when it is set), and that with onlyLongestMatchIgnoreSubwords="false" the filter now emits every dictionary match. Optionally fix the 10.2.0 CHANGES.txt entry, which names the option onlyLongestMatchNoSubwords while the factory key is onlyLongestMatchIgnoreSubwords.

Trade-offs

Documentation only. Not run.

Keep a notice, with its own message, for the one configuration whose output changed in 11.0, or leave the exact condition to maintainers.

Trade-offs

A design choice beyond this bug; it adds a second warning that later releases would have to retire. Not run.

Method: non-formal (code reading, then executed tests). The hunt unit checked the analysis SPI chain: for each in-scope factory, a static inventory of its get/getBoolean reads was compared with the arguments the factory passes to its filter constructor. In this factory the onlyLongestMatch value reaches no constructor and only guards the warning. A public-API JUnit with java.util.logging capture reproduced it on the stock main jar, and a release matrix dated it to #14356. The finder's first framing (an explicit false being "silently ignored") was rejected as intended by the verifiers and the adjudication, who re-scoped the bug to the warning condition; it was then cut to the one-test repro.