Hunspell spell-checker can miss an x|xx triple letter at the second and later joins of a compound

CHECKCOMPOUNDTRIPLE tells Hunspell to reject a compound when a join puts three identical letters in a row, either two before and one after the join (foo|ox) or one before and two after (xo|oof). Lucene's Hunspell.spell splits a compound left to right and checks each join. From the second join on, the code passes the join as a position in the whole word but the length of only the part still being split, and the "one before, two after" half of the test compares the two (breakPos < length - 1, Hunspell.java:430), while the other half adds the part's offset. Once enough letters precede the current part, that half is skipped. With Lucene's own test dictionary, bareeel is rejected but operabareeel, foobareeel and barebareeel are accepted; with the public Swedish dictionary, sommarmusikradiooomskuren is accepted. C Hunspell rejects all of them. Only the opt-in spell-checker (spelling verdicts and suggestions) is affected, in every release from 8.9.0 to 10.6.0 (10.6.0 by reading); the fix is one line.

• Compound checking: when a word is not in the dictionary, Hunspell.spell tries to split it into dictionary words that may be compounded (here, words with the COMPOUNDFLAG). It tries a first part, then splits the rest the same way, recursively, so a word can have several parts (COMPOUNDWORDMAX, the part cap, defaults to unlimited, Dictionary.java:152; the compound flags or rules decide which words may start, continue or end a compound).
• CHECKCOMPOUNDTRIPLE (hunspell(5)): "Forbid compounding, if compound word contains triple repeating letters (e.g. foo|ox or xo|oof)." Two shapes: xx|x, two equal letters before the join and one after (foo|opera), and x|xx, one before and two after (bare|eel).
• The part being split: from the second join on, the code works on the rest of the word as a slice of the same array: an offset (where the rest starts) and a length (how many letters it has).

1. checkCompounds computes the join as a position in the whole word, breakOffset = word.offset + breakPos (Hunspell.java:223), and calls mayBreakIntoCompounds(word.chars, word.offset, word.length, breakOffset) (:224): offset and length of the slice, position in the whole word.
2. After a part is found, checkCompoundsAfter recurses on the rest, new CharsRef(word.chars, breakOffset, remainingLength) (:284), so from the second join on the offset is greater than 0.
3. mayBreakIntoCompounds (:427-433) tests breakPos > offset + 1 && chars[breakPos - 2] == chars[breakPos - 1] for xx|x (offset-aware) || breakPos < length - 1 && chars[breakPos] == chars[breakPos + 1] for x|xx (:430): a position in the whole word compared with the slice's own length.
4. Whenever the join's position is at least the slice length minus 1, the x|xx test is skipped and the join is allowed. The buggy bound is stricter than the right one (offset + length - 1), so it never reads out of bounds; it only skips the check.

With p letters before the part being split and a letters after the join, the x|xx test runs only when p < a - 1. At the first join p = 0, so it always runs.

The same bare|eel join is rejected in foobareeelgrass and accepted in operafoobareeelgrass: only the prefix differs. C Hunspell recurses on the substring (affixmgr.cxx:2084), so both halves of its test use positions inside the rest (affixmgr.cxx:1823-1827).

• Who. Callers of the spell-checker API: Hunspell.spell, and the suggesters, which validate candidates with the same check (Suggester.java:215, :298; ModifyingSuggester.java:362-396; GeneratingSuggester.java:323). A probe on main with the test dictionary: suggestNoTimeout("operabareeell") returns [operabareeel], the misspelled word, and [] with the fix.
• Opt-in twice. The spell-checker is a separate API that no analyzer calls, and the dictionary must set CHECKCOMPOUNDTRIPLE and allow compounds of 3 or more parts (COMPOUNDWORDMAX sets no cap by default; whether a middle part is allowed depends on the dictionary's compound flags or rules).
• Real dictionaries. Among 191 public .aff files surveyed (local copies of the LibreOffice dictionaries and wooorm/dictionaries), CHECKCOMPOUNDTRIPLE appears in hu_HU, nb_NO and sv_SE/sv_FI. Per the finder's survey, hu_HU allows only 2-part compounds and nb_NO has no compoundable root that starts with a doubled letter. In sv_SE the finder's targeted search found 20 accepted triple words, all an oo- word (oomskuren, oomstridd, oomkullkastlig, oomkullrunkelig) after a part ending in o, such as sommarmusikradiooomskuren and sommarpolisradiooomskuren; the independent reproducer re-ran the first, the devil's advocate both. sv_FI and nb_NO were not run.
• Effect. A misspelled compound is reported as correct, silently. Correct words are not affected: the bug only skips a rejection. In sv_SE the simplified spelling sommarmusikradioomskuren is rejected by both C Hunspell and Lucene, so only the triple form is newly accepted.
• Not affected. HunspellStemFilter uses Stemmer (HunspellStemFilter.java:86, :110-111) and getRoots returns no roots for compounds (Hunspell.java:300); executed: getRoots("operabareeel") is [] and the filter emits the same tokens with and without the fix. No index or query test was run; the rest follows from the code path. A code search of Elasticsearch, OpenSearch and Solr (which can miss hits) found only the stem filter.
• Likelihood: rare. It needs such a dictionary, a compound of 3 or more parts, a later part that starts with a doubled letter equal to the previous part's last letter, and enough letters before it (p >= a - 1). No scale is needed: one word triggers it.

# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
    => java.lang.AssertionError: spell("operabareeel") expected:<false> but was:<true>
       org.junit.Assert.fail(Assert.java:89)
       org.junit.Assert.failNotEquals(Assert.java:835)
       org.junit.Assert.assertEquals(Assert.java:120)
       org.apache.lucene.analysis.hunspell.TestCheckCompoundTripleLaterBoundary.testTripleLetterAtSecondBoundary(TestCheckCompoundTripleLaterBoundary.java:36)

• Contract. hunspell(5): "CHECKCOMPOUNDTRIPLE Forbid compounding, if compound word contains triple repeating letters (e.g. foo|ox or xo|oof)." It sets no limit on the number of parts, and "COMPOUNDWORDMAX ... (Default is unlimited.)". Hunspell.spell returns "whether the given word's spelling is considered correct according to Hunspell rules" (Hunspell.java:71-74). The class "can be used in place of native Hunspell for many languages" (:35-36) and notes that "not all languages are supported yet. For example:" Hungarian, non-BMP characters, the PHONE option (:36-44). Those are examples, not a full list, so they neither include nor exclude this case; nothing in the javadoc, tests or history marks multi-part CHECKCOMPOUNDTRIPLE as unsupported.
• Tests. Lucene's fixture checkcompoundtriple.wrong lists fooopera and bareeel, run by TestSpellChecking.testCheckCompoundTriple (TestSpellChecking.java:132-134). The fixture, the same as C Hunspell's, has only 2-part compounds, so it never reaches a later join. No test asserts that a later-join triple is accepted.
• Code. Two halves of one boolean expression use different index spaces, and the verdict for the same join flips with the prefix length. C Hunspell, which the code ports, uses positions inside the rest in both halves; Lucene's breakPos > offset + 1 is the port of its i > 1.
• History. Both halves were written in one commit (9d45dfe776a2, LUCENE-9710, 2021-02-01); its PR had no review comments, and the issue has no design discussion. CHANGES 8.9.0 (LUCENE-9687) says the work fixed "various behavior differences between Java and C++ implementations" (lucene/CHANGES.txt:5368-5369).
• Reference. C Hunspell 1.3.2 and 1.7.2 reject all 9 triple words of the independent reproducer's set on the same dictionary.

The strongest case: the class promises to replace native Hunspell only for many languages, and both the manual's examples and the fixtures show only 2-part compounds, so behaviour for 3 or more parts is unspecified.

Why it fails: the manual states the rule for any compound and sets no part limit; COMPOUNDWORDMAX defaults to unlimited; Lucene enforces the other half (xx|x) at every join, so it treats the rule as applying everywhere; the javadoc's examples of unsupported cases do not name this directive or Swedish (they are only examples, so this point is weak on its own); and nothing explains why the same join is accepted or rejected depending only on how many letters precede it. The fix does change established spell and suggest results, a compatibility cost noted under the fix, not a sign that the current results are intended. Every reviewer rated the intended-behaviour case weak. Verdict: not intended, a defect.

One fix was built and run in a scratch worktree at a9779ef3c193; it matches C Hunspell's test and the other half of Lucene's. For the regression test, the 3 fixture lines are the smallest change (the fixture then has words C Hunspell's copy lacks, which C also rejects); a dedicated test class like the primary repro states the intent more clearly. Test scope: the runs cover flag-based recursive compounding (COMPOUNDFLAG, COMPOUNDBEGIN/MIDDLE/END); combinations with CHECKCOMPOUNDPATTERN, COMPOUNDRULE or SIMPLIFIEDTRIPLE were not tested for exact parity with C Hunspell (whose triple test also has scpd == 0 && !words guards, affixmgr.cxx:1823-1827, and a second copy at :2418-2422).

Compare the absolute position with the part's end, offset + length, as the xx|x half does with its start. chars[breakPos + 1] stays inside the part.

if (breakPos > offset + 1 && chars[breakPos - 2] == chars[breakPos - 1]
-    || breakPos < length - 1 && chars[breakPos] == chars[breakPos + 1]) {
+    || breakPos < offset + length - 1 && chars[breakPos] == chars[breakPos + 1]) {
  return false;
}

Trade-offs

Changes verdicts only for CHECKCOMPOUNDTRIPLE dictionaries, and only to reject words with an x|xx triple at a later join, which C Hunspell rejects too. In the sv_SE words measured, rejected words do the same or fewer lookups than on main (436 vs 465 and 1,371 vs 1,420), and words main wrongly accepts now run the full search before they are rejected (1,350 vs 290 lookups); all took about 4 ms or less per word. Not benchmarked further. Compatibility: it deliberately changes what the public spell() and the suggesters return for such words (operabareeell no longer gets the suggestion operabareeel); applications or tests that rely on the old acceptance would see that, so it is a narrow behaviour fix to note in CHANGES.

What the checks validated

• Scratch worktree at a9779ef3c193, gradle, seed DEADBEEF: stock plus the fixture lines fails 4 of 60 (the two repro classes and TestSpellChecking), with assertions on and off.
• With the fix and the fixture lines: --tests 'org.apache.lucene.analysis.hunspell.*' plus the two repro classes, 48 classes, 165 tests, 0 failures, 13 skipped (tests that need external dictionaries), with assertions on and off.
• :lucene:analysis:common:check -x test (google-java-format, ecj lint, forbidden APIs) passes with the fix and the fixture lines.
• Earlier, with the fix shadowed on ea57360912ed: all repro runs pass (finder 2/2, independent reproducer 5/5, cross-model stage 4/4, judge re-run 5/5), Lucene's hunspell test classes 146/146, and the introducing commit plus the fix rejects every triple.

Give mayBreakIntoCompounds the break inside the part and index chars[offset + i], so both halves read i > 1 and i < length - 1.

Trade-offs

Intended to be equivalent to A by inspection, with a larger diff (the caller and the CHECKCOMPOUNDCASE lines change too). No patch was written and nothing was run.

Method: differential testing against another implementation (the hunt counts this as testing, not as checking against an exact reference semantics).

• A generator built word lists from the 131 Hunspell 1.7.2 conformance fixtures: fixture words, every generated form, case variants, edits, all 2-part and sampled 3-part concatenations of roots and forms. A small C driver called Hunspell_spell per word, and the results were compared with the stock Lucene jar.
• For the checkcompoundtriple fixture, 4 three-part words were accepted only by Lucene: operabareeel, foobareeel, eelbareeel, barebareeel.
• Code reading of mayBreakIntoCompounds found the mixed index spaces; the finder reduced it to a public-API JUnit and searched the public sv_SE dictionary for accepted triple words (20 found).
• An independent reproducer, a cross-model stage with its own executed test, a devil's advocate and an adjudication then confirmed it (see the checks).