Hunspell: a .dic entry with an undefined AM morphological alias throws ArrayIndexOutOfBoundsException instead of a descriptive error

Hunspell affix files can number their morphological descriptions in an AM table ("alias compression"), and .dic entries then carry only the number. When an entry uses a number the table does not define, Dictionary.splitMorphData reads morphAliases[alias - 1] with no range check and catches only NumberFormatException. So AM 1 / AM po:noun plus the entry foo<TAB>2 makes every Dictionary constructor throw ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1; alias 0 gives Index -1, and an AM table with fewer lines than its count gives a NullPointerException. The flag-alias lookup a few lines above turns the same out-of-range lookup into IllegalArgumentException: Bad flag alias number:N, and has since 2012 (LUCENE-4079); C Hunspell's source range-checks the index. The unchecked lookup came with AM support itself (LUCENE-5778, 4.10.0) and is in every release since, including 10.6.0 and main. The malformed dictionary is refused either way: there are no wrong results, only an unhelpful error. A one-line range check mirroring the AF guard fixes it.

• AM table (morphological aliases): a header AM <count> and then one AM <fields> line per alias, numbered from 1. hunspell(5): "Hunspell can also substitute morphological data with ordinal numbers in affix rules (alias compression)".
• AF table (flag aliases): the same idea for flag sets; foo/2 means the flags of AF alias 2.
• morphAliasCount: how many AM entries the .aff actually defined.

1. parseMorphAlias (Dictionary.java:1295-1304 at 645d4fe14a20) allocates morphAliases = new String[count] from the declared count and fills one slot per AM line, so slots beyond the lines actually given stay null.
2. splitMorphData (:1306-1315) parses a numeric morphological field and does morphData = morphAliases[alias - 1] (:1311) inside a try that catches only NumberFormatException (meant for non-numeric, literal fields). There is no check of alias against 1 and morphAliasCount.
3. An alias above the declared count, or 0 or below, throws ArrayIndexOutOfBoundsException; an alias in a declared but undefined slot yields null and a NullPointerException at morphData.isBlank() (:1315).
4. Both loading passes reach it (mergeDictionaries :1030 while probing for morphological data, and readMorphFields :1142/:1207 for every entry with morphological fields while the word table is built), with either sorting strategy, and the exception leaves the constructor.
5. The flag-alias sibling getAliasValue (:1287-1293) catches the IndexOutOfBoundsException and throws IllegalArgumentException("Bad flag alias number:" + id, ex).

.aff SET UTF-8, AM 1, AM po:noun; .dic 1, foo<TAB>2: ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 at Dictionary.splitMorphData(Dictionary.java:1311), from mergeDictionaries(:1030) and the constructor (:268). foo<TAB>0: Index -1 out of bounds for length 1; foo<TAB>-1: Index -2. AM 2 with only AM po:noun, entry foo<TAB>2: NullPointerException: Cannot invoke "String.isBlank()" because "morphData" is null. Control: AF 1, AF A, entry foo/2: IllegalArgumentException: Bad flag alias number:2.

• Who is hit. Users of the opt-in hunspell analysis component whose dictionaries use AM aliases, through new Dictionary(...) (all constructors, both sorting strategies) or HunspellStemFilterFactory (SPI name hunspellStem, for example in CustomAnalyzer).
• What it takes. One .dic entry whose morphological field is a number the AM table does not define. No upstream test dictionary (Lucene's morphalias; C Hunspell's alias, alias2 and alias3 tests) or report shows one, and public dictionaries were not surveyed, so it is rated rare.
• What happens. One bad word entry fails the whole load, with an exception that does not say which alias or input line is wrong: "Index 1 out of bounds for length 1" means "alias 2 is not in your AM table". HunspellStemFilterFactory.inform wraps only ParseException into its Unable to load hunspell data! [dictionary=...,affix=...] IOException (HunspellStemFilterFactory.java:115-118), so factory users do not even get the file names.
• What does not happen. No wrong stems, no silent acceptance, no hang, nothing written. Correct dictionaries, dictionaries without an AM table, and dictionaries whose AM header has no AM lines (the lookup is skipped while morphAliasCount is 0, Dictionary.java:1308) do not hit it.
• Related, separate fix sites (not part of this bug). An AF table that declares more aliases than it defines gives NullPointerException ... "rawFlags" is null; AF/AM tables with more lines than their count, or a missing or non-numeric count, throw raw exceptions while the .aff is parsed. With the offline sorting strategy this exception is also one trigger of LFV-1006 (temp output not released on failure).

The oracle does not pin the fix's exception type: loading foo (what C Hunspell's source does), a ParseException (what the constructors document) or an IllegalArgumentException (what the AF table gives) all pass. It needs SortingStrategy, so it compiles on 9.9.0 and later; older releases were checked with a release-agnostic probe that calls the public constructor of each version.

As the regression test: the same test, for example as a new method in TestDictionary, with the unused catch parameter descriptive renamed to _ (Lucene's ecjLintTest rejects unused exception parameters; checked with gradle). With the recommended fix it can be tightened to expectThrows(IllegalArgumentException.class, ...). It pins the beyond-the-table case only; that the fix also covers alias 0 and the NullPointerException case was shown by probes with the fix.

cp TestDictionaryUndefinedMorphAlias.java $LUCENE/lucene/analysis/common/src/test/org/apache/lucene/analysis/hunspell/
cd $LUCENE && ./gradlew :lucene:analysis:common:test --tests org.apache.lucene.analysis.hunspell.TestDictionaryUndefinedMorphAlias -Ptests.seed=DEADBEEF
# assertions off: add -Ptests.asserts=false (same result)

# gradle, main 645d4fe14a20, seed DEADBEEF (assertions on; the same with -Ptests.asserts=false)
TestDictionaryUndefinedMorphAlias > testMorphAliasBeyondAmTable FAILED
    java.lang.AssertionError: expected a descriptive error for undefined AM alias 2 (the table has 1 entry); actual: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:36D1E7B404813866]:0)
        at org.apache.lucene.analysis.hunspell.TestDictionaryUndefinedMorphAlias.testMorphAliasBeyondAmTable(TestDictionaryUndefinedMorphAlias.java:40)
        Caused by:
        java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1
            at org.apache.lucene.analysis.hunspell.Dictionary.splitMorphData(Dictionary.java:1311)
            at org.apache.lucene.analysis.hunspell.Dictionary.mergeDictionaries(Dictionary.java:1030)
            at org.apache.lucene.analysis.hunspell.Dictionary.<init>(Dictionary.java:268)
            at org.apache.lucene.analysis.hunspell.TestDictionaryUndefinedMorphAlias.testMorphAliasBeyondAmTable(TestDictionaryUndefinedMorphAlias.java:36)

# with the recommended range check: the test passes (assertions on and off)

# release jars 10.6.0 (assertions on and off) and 9.9.0 (off): same failure, lookup at Dictionary.java:1323 and :1278

• Sibling guard. getAliasValue (Dictionary.java:1287-1293) wraps the out-of-range AF lookup into IllegalArgumentException("Bad flag alias number:" + id, ex); it came with AF support in LUCENE-4079 (b4fe59532ee, 2012-05-30) and LUCENE-5468 carried it into the current class. LUCENE-5778 (532d9f246b8, 2014-06-21) added AM with morphAliases[alias-1] and a NumberFormatException catch commented // fine; its issue never discusses undefined aliases.
• Contract. @throws ParseException Can be thrown if the content of the files does not meet expected formats (:187, :204, :224). No javadoc, test, MIGRATE.md or CHANGES.txt entry mentions an index exception for malformed content.
• Maintainer intent. LUCENE-3976 (#5049), Chris Male, 2012-06-01: "I think we should explore what those possible errors are and fix them at their source, to provide targeted Exceptions. If there is a problem parsing, then we should thrown a ParseException with the line number causing the problem", then "We will tackle error messages on a per-error basis". Later work went the same way: #12468 (stricter .aff checks with ParseException) and #16919 (merged 2026-10-28: it replaced an AssertionError for a malformed affix condition with an IllegalStateException and stopped catching Throwable in AffixCondition).
• Tests. TestDictionary.testInvalidData expects ParseException for malformed affix rules; no hunspell test uses an undefined AM alias, and none expects an index exception.
• Reference implementation. C Hunspell's HashMgr::get_aliasm checks index > 0 && index <= aliasm.size() and otherwise returns null; its error: bad morph. alias index: %d warning prints only in builds with HUNSPELL_WARNING_ON.

The strongest case for "intended" (an intended-behaviour checker, and a read-only reviewer from another model family, OpenAI GPT-5.6, which leaned "not real"): the input is a malformed, trusted configuration file and is refused loudly either way, with no wrong results; "can be thrown" is permissive, not a promise that every malformed file gives ParseException; the parser already lets several unchecked exceptions escape for nearby malformed content (the AM/AF header parsers); and upstream files such changes as improvements (LUCENE-3976 was an enhancement, #12468 is listed under Improvements).

Why it fails: it limits severity, not validity. Nothing presents an index exception as intended; the same mistake one table over is deliberately turned into a named error; and the maintainer's stated policy was to fix such errors at their source with targeted exceptions. The other raw exceptions are separate, equally unpolished sites, not evidence of a design. Hence "wrong error" at low priority: the case is for a descriptive error, not a crash on valid input.

Two options were run through gradle in a scratch worktree of 645d4fe14a20; A is recommended.

In Dictionary.splitMorphData (Dictionary.java:1310-1311).

         int alias = Integer.parseInt(morphData.trim());
+        if (alias < 1 || alias > morphAliasCount) {
+          throw new IllegalArgumentException("Bad morphological alias number:" + alias);
+        }
         morphData = morphAliases[alias - 1];

Trade-offs

• The load still fails, now with the same exception type and wording as the AF case.
• Comparing with morphAliasCount (entries defined), not the array length, also covers alias 0, negative aliases and the NullPointerException case.
• No line number: splitMorphData has no reader; a ParseException with a line would need the callers to track lines and rethrow.
• IllegalArgumentException is not a ParseException, so HunspellStemFilterFactory still adds no file names, as for AF today. No signature or file-format change, but the exception callers see for this malformed input changes; one comparison per numeric morphological field at load time.

What the checks validated

• Gradle, scratch worktree of 645d4fe14a20: without the fix the repro fails with assertions on and off; with it, it passes with assertions on and off.
• Closest suites: every test class of the hunspell package (TestDictionary, TestMorphAlias, TestMorphData, TestMorph, TestHunspellStemFilterFactory, TestSpellChecking, TestStemmer, ...): 47 classes with the repro, 159 tests, 0 failures, 13 skipped (three classes that need external dictionaries or corpora, or are @Ignored).
• ./gradlew :lucene:analysis:common:check -x test passes with the fix and the test in place (google-java-format, ecj lint, forbidden APIs, checksum checks) once the test's unused catch parameter descriptive is renamed to _; with the original name ecjLintTest rejects it. The renamed test still fails on stock and passes with the fix. The full test suite was not run.
• Earlier, shadowed into the stock main jars: the finder's 5-test JUnit passes 5/5 with 79 upstream hunspell tests, the reproducer's 4-test JUnit 4/4, and the minimal test with assertions on and off.

The same check, but return Collections.emptyList(); instead of the throw.

         int alias = Integer.parseInt(morphData.trim());
+        if (alias < 1 || alias > morphAliasCount) {
+          // like C Hunspell, which warns "bad morph. alias index" and keeps the word
+          return Collections.emptyList();
+        }
         morphData = morphAliases[alias - 1];

Trade-offs

The dictionary loads and foo has no morphological data, as in C Hunspell's source. But the mistake goes unnoticed: Dictionary has no warning channel, and C Hunspell's own warning prints only in builds with HUNSPELL_WARNING_ON; and it departs from Lucene's choice for the same mistake in AF, which fails.

What the checks validated

Gradle in the same scratch worktree: the repro passes with assertions on, and the hunspell package runs without failures (47 classes, 159 tests, 0 failures, the same 13 skips). Assertions off and the static checks were not run for B.

The getAliasValue pattern.

Trade-offs

Fixes the ArrayIndexOutOfBoundsException cases but not the NullPointerException one: declared but undefined slots are inside the array and hold null. Not tried.

Method: non-formal (code reading, then executed tests). On 2026-10-29 the hunt unit for analysis loaders was modelling, in TLA+, whether the hunspell loader releases its temporary sort output when loading fails (the sibling LFV-1006). To list the content-driven exceptions that can make the dictionary-merge step throw, it read the parsing code and noticed that splitMorphData reads morphAliases[alias - 1] unchecked while getAliasValue guards the same lookup. The TLA+ model has no alias logic; the defect was read off the code. A probe (seven alias-table cases, including the NullPointerException variant and the AF neighbours) and a 5-test public-API JUnit confirmed it; independent checkers then reproduced it with their own tests, and it was cut to the one-test repro.