Hunspell checks a short `[ ]` or `.` affix condition at the wrong end of the strip

A Hunspell affix rule has a strip, an affix and a condition: SFX A um a [m] means "remove um, add a, if the root ends in m". When a condition contains [ ], . or - and has no more character patterns than the strip has letters, it lies wholly inside the strip, and AffixCondition.compile decides the rule once, at load: it matches the strip against kind == PREFIX ? ".*" + condition : condition + ".*" (AffixCondition.java:107). That tests a suffix condition at the strip's start and a prefix condition at its end, the opposite of Hunspell. So the rule never applies when it should (data from datum rejected, redo from undo rejected) or always applies when it should not (quanta accepted with root quantum, repack stemmed to unpack). Hunspell.spell, getRoots, the Stemmer and HunspellStemFilter give these wrong answers with no error, while the same condition written as a plain letter (m) works. C Hunspell 1.3.2 gives the expected answers. It is a regression from LUCENE-9806 (3a99e2aa82d, 2021), present in every release from 8.9.0 to 10.6.0 (10.6.0 by reading) and on main; the fix swaps the two templates. None of the 191 surveyed public dictionaries changes its results.

• Affix rule: a line SFX flag strip affix condition (or PFX) in the .aff file. To make a word from a root, remove the strip from the root's end (start for PFX) and add the affix, but only if the root meets the condition. SFX A um a [m] turns datum into data.
• Condition: a pattern of single characters: a letter, . (any character) or [...] (a set; [^...] its complement). hunspell(5) explains its own example SFX B y ied y as "can be added to the words terminated with an `y'": a suffix condition is matched against the root's last characters, which include the strip.
• Lucene's shortcut: when spelling or stemming, Lucene sees the word with strip and affix already removed, so AffixCondition.compile splits the condition at load into the part that falls on the strip, checked once, and the part that falls on the remaining stem.

1. A condition that contains [, . or -, that uniqueKey has not already mapped to "always true" (it does so for . alone and for a condition that literally equals the strip's start or end, AffixCondition.java:73-80), and that has no more character patterns than the strip lies wholly on the strip (:105-106).
2. Lucene then decides the rule once: String regex = kind == PREFIX ? ".*" + condition : condition + ".*"; and strip.matches(regex) ? ALWAYS_TRUE : ALWAYS_FALSE (:107-108).
3. condition + ".*" matches the strip's start, which is right for a prefix rule, but it is given to suffix rules; ".*" + condition matches the strip's end and is given to prefix rules.
4. For conditions in Hunspell's syntax (letters, . and [...], one character each) both templates agree when the condition is as long as the strip, so only shorter conditions get the wrong constant. Every consumer then uses that constant: Stemmer (for spell, getRoots and HunspellStemFilter), word-form generation and suggestions.

The same condition gets different answers depending only on how it is spelled: for SFX abc d <cond> on the root xabc, stock main accepts xd with c, bc, abc and [a]bc and rejects it with [c], [b]c and .c; with the fix all 7 spellings (and the 7 prefix ones) agree.

A lone dash takes the same path: Lucene treats any condition with - as a pattern (AffixCondition.java:153-155), while hunspell(5) says "Dash hasn't got special meaning". So SFX A a- x - on the root ba- rejects bx on main, while C Hunspell 1.3.2 gives bx with the root ba-, and so does Lucene with the fix.

• Who. Users of Lucene's Hunspell analysis: the spell checker Hunspell (spell, getRoots, suggestions) and HunspellStemFilter / HunspellStemFilterFactory. No default analyzer uses Hunspell.
• Two halves. A condition that is redundant with the strip (SFX A um a [m]) is never applied, so correct words are rejected and stems are lost; it is natural to write, and C Hunspell 1.3.2 applies it with no warning, also in 8-bit mode. A condition that contradicts the strip (SFX B um a [u]) is always applied, so wrong words are accepted and wrong stems appear; C Hunspell in 8-bit mode warns "incompatible stripping characters and condition" for such rules, so they are unusual (the 1.7.2 source skips that check in UTF-8 mode).
• Real dictionaries. 191 public .aff files (99 from LibreOffice, 92 from wooorm/dictionaries, 2,306,338 rules) contain 4 rules of this shape, all in the LibreOffice bn_BD folder. Their bracket holds both characters of the strip, so both ends agree and no result changes. The survey counted conditions with [, . or -. None of the surveyed dictionaries is affected; others were not checked.
• Effect. Silent wrong spelling verdicts, missing stems (data stays data) and invented ones (repack becomes unpack, so on a stemmed field the query unpack matches repack). HunspellStemFilter also runs at index time, so the indexed stems of such words change with the fix.
• Scale. None needed: one word triggers it, deterministically, every time such a dictionary is loaded.
• Versions. Every release from 8.9.0 to 10.6.0 (10.6.0 by reading), and main.

# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
    => java.lang.AssertionError: spell("xd") expected:<true> but was:<false>
       org.junit.Assert.fail(Assert.java:89)
       org.junit.Assert.failNotEquals(Assert.java:835)
       org.junit.Assert.assertEquals(Assert.java:120)
       org.apache.lucene.analysis.hunspell.TestAffixConditionShorterThanStrip.testSuffixConditionAtEndOfRoot(TestAffixConditionShorterThanStrip.java:36)

• Contract. hunspell(5): the condition "must be met before the affix can be applied", [...] is a character set and . any character; the example SFX B 0 ed [^y] / SFX B y ied y is explained as "First B suffix can be added to a word if the last character of the word isn't y'. Second suffix can be added to the words terminated with an y'" (C Hunspell 1.7.2 man/hunspell.5:88-89, :95-98, :541-544). The man page does not state "the end of the root" as a rule in so many words; its example and the C code fix the alignment: SfxEntry::checkword builds stem + strip and tests the condition backwards from the end, PfxEntry::checkword builds strip + stem and tests forwards (affentry.cxx:232-243, :651-667).
• Lucene's own javadoc. acceptsStem: "whether the given word matches this condition as a stem with both "strip" and "affix" removed" (AffixCondition.java:63-66). Package: "A Java implementation of Hunspell stemming and spell-checking algorithms". Hunspell: "can be used in place of native Hunspell for many languages"; its examples ("For example:") of what is not supported yet are Hungarian, characters outside the Basic Multilingual Plane and the PHONE option (Hunspell.java:35-44). CHANGES 8.9.0, LUCENE-9687: "fix various behavior differences between Java and C++ implementations".
• Lucene contradicts itself. uniqueKey aligns literal conditions the Hunspell way (strip.endsWith(condition) for SFX, strip.startsWith(condition) for PFX, AffixCondition.java:73-80), asserted by TestAffixCondition.testUniqueKey (uniqueKey(SUFFIX, "abc", "bc") is always true); the branch for conditions longer than the strip does too (:111-123). So m and [m], or c, [c] and .c, give different answers.
• History. Before 3a99e2aa82d (LUCENE-9806, "speed up affix condition checking", 2021-02-24), Dictionary built isSuffix ? ".*" + condition : condition + ".*" and ran it on the whole root, which is correct. The commit moved the check onto the strip alone and kept the two templates with the kinds swapped. Its PR description says "No new tests"; the review is about regex and automaton building. The only bracket case in TestAffixCondition that reaches this branch has as many patterns as the strip, where both templates agree.
• Tests. None asserts the current behaviour: Lucene's test fixtures have one short regexp rule, where both alignments agree; C Hunspell 1.7.2's tests have none. Every hunspell test that runs passes with the fix (13 are skipped because they need external dictionaries).

Strongest case (the intended-behaviour checker, the cross-model reviewers and the adjudication): a condition that lies wholly inside the strip is either redundant or contradicts the strip, so such rules are odd dictionary input; the man page does not literally say "the end of the root"; HunspellStemFilter is @lucene.experimental; no public dictionary is affected.

Why it fails: the redundant case (SFX A um a [m]) is ordinary, and C Hunspell 1.3.2 applies it with no warning; Dictionary loads both kinds of rule without a ParseException; the man page's own example and the C code fix the alignment; Lucene's literal path and longer-condition path use that alignment, so m and [m] disagree, which no reading of the format allows; Hunspell and Dictionary are not experimental, and "experimental" is about API stability, not wrong results; rarity bears on impact, not on intent. Every reviewer rated the intended-behaviour case weak. Verdict: a defect.

One fix was built and validated in a scratch worktree; the second option is an equivalent rewrite that was not run. A related bug in the same lines (dash ranges in the strip part) has its own fix and can share a PR.

In AffixCondition.compile, the branch for a condition that lies inside the strip (AffixCondition.java:106-109):

       if (conditionChars <= strip.length()) {
-        String regex = kind == PREFIX ? ".*" + condition : condition + ".*";
+        // the condition lies inside the strip: it must match the strip's start (PFX) or end (SFX)
+        String regex = kind == PREFIX ? condition + ".*" : ".*" + condition;
         return strip.matches(regex) ? ALWAYS_TRUE : ALWAYS_FALSE;
       }

Trade-offs

• None found for cost: the branch still runs once per distinct rule at load and still returns a constant.
• For conditions in Hunspell's syntax it changes results only for conditions shorter than the strip (a condition with other Java regex characters, outside the format, can change at equal length too: .*b on the suffix strip abc goes from always true to always false). Dictionaries with such rules accept and stem differently after the upgrade, so fields stemmed with them need reindexing.
• The dash-range handling in the same lines ([a-c] read as a range) is a separate bug with its own fix; a lone - condition is this bug and the swap fixes it.

What the checks validated

• Gradle, scratch worktree at a9779ef3c193, seed DEADBEEF, assertions on and off. Stock, with the unit test and the four repro classes: 31 tests, 20 failures (the paste-ready test 3 of 4, the finder's 4 of 5, the independent reproducer's 10 of 12, the spellings test 2 of 2, the new unit test); every control passes.
• With the fix, --tests 'org.apache.lucene.analysis.hunspell.*': 50 classes, 182 tests, 0 failures, 13 skipped (they need external dictionaries); that is Lucene's 46 hunspell test classes with 159 tests, including the new unit test, plus the 4 repro classes.
• :lucene:analysis:common:check -x test with the fix, the unit test and the paste-ready test passes (formatting, ecj lint, forbidden APIs).
• Repro harness: the paste-ready test fails 3 of 4 on main (assertions on and off), 10.5.2 and 9.12.3, and passes 4 of 4 with the fix. A probe of the lone-dash rule SFX A a- x -: bx rejected on main, accepted with the fix, as C Hunspell 1.3.2 answers.
• Earlier, on ea57360912ed: the finder's test 5 of 5, the independent reproducer's 12 of 12 and upstream TestAffixCondition 7 of 7 and the 146 tests of Lucene's hunspell test classes pass with the swap; the bounded-exhaustive check of the real class (144,482,592 cases) loses its 4 wrong-end classes; 3a99e2aa82d plus the swap is correct.

For SUFFIX, match the strip's last conditionChars characters against the condition (strip.substring(strip.length() - conditionChars).matches(condition)); for PREFIX, the first ones.

Trade-offs

For conditions made of Hunspell's one-character patterns (letters, ., [...]) this gives the same answers as the swap and states the intent more plainly, at a slightly larger diff; conditions with other regex characters could behave differently. Not run.

Method: semi-formal (a hypothesis from code reading while modelling, witnessed by the model and by a bounded-exhaustive check of the real code). Hunt unit on Hunspell, main ea57360912ed, 2026-10-26.

• SMT model (Z3) of compile's split of a condition into a strip part and a stem part, against the hunspell(5) semantics: mismatches are satisfiable only when the condition has fewer character patterns than the strip, for both prefix and suffix rules; unsatisfiable for equal or longer conditions and for the corrected model.
• Bounded-exhaustive translation validation of the real AffixCondition on the stock jar: 40,494 conditions, strips up to 3 and stems up to 2 characters, 144,482,592 checks. 4 wrong-end classes for shorter conditions, gone with the swap (the remaining dash-range classes are the separate bug; with both fixes 0 checks differ).
• JUnit on the public API, then C Hunspell 1.3.2 as the oracle, independent reproduction, cross-model reviews, a devil's advocate and an adjudication (see the checks below).
• Survey of 191 public dictionaries for rules of this shape (4 found, none changing a result).