suggestSimilar sizes its candidate search as numSug × maxInspections in an int. An "unlimited" maxInspections times an even count wraps to a negative budget: the two-edit pass is then skipped (an empty result) or the one-edit pass reads the top of an empty queue (a NullPointerException).
suggestSimilar(term, numSug, reader)returns up tonumSugindexed words that are close to a misspelled word by edit distance, best first.- It first collects candidates one edit away. Only if it found fewer than its budget does it also look two edits away.
- The budget is numSug × maxInspections: with the default 5 and 2 suggestions it keeps the best 10 candidates.
setMaxInspectionsraises it; its javadoc says this "can improve the accuracy of results, at the cost of performance".
2 * 2147483647 wraps to -2 (odd counts stay positive). With a one-edit neighbour the call throws NullPointerException; for luxeny, whose only neighbour lucene is two edits away, it returns [] with no error. Select the still to open it at full size. Generated with AI, checked against the code.- Index one word,
lucene. Create aDirectSpellCheckerand callsetMaxInspections(Integer.MAX_VALUE), the largest accepted value; no upper bound is documented (the default is 5; the javadoc says raising it "can improve the accuracy of results"). - Ask for suggestions for
luxeny, two edits fromlucene. numSug 1 returns[lucene]. You expect numSug 2, "up to 2 suggestions", to return[lucene]too. - Instead you get
[]and no error. The candidate budget is numSug × maxInspections in anint: 2 × 2,147,483,647 = 4,294,967,294, which wraps to -2. The two-edit pass runs only while the hits found are below the budget, and 0 < -2 is false. - With a one-edit neighbour (index
lucene,lucent,lucerne; wordlucenx) the same call throwsNullPointerException: the first pass treats its empty queue as full (0 >= -2) and reads the top of an empty queue. - Small odd counts work (the budget becomes 2^31 - numSug, still positive and large), so 1 and 3 succeed while 2 and 4 fail. At the default 5 only absurd counts wrap, from numSug 429,496,730 on; numSug 858,993,460, for example, returns 4 of 10 suggestions.
Applications that set an "unlimited" maxInspections (Integer.MAX_VALUE, 2^30, Integer.MAX_VALUE - 1) and ask for an even number of suggestions (with 2^30, every count except 1, 5, 9, ...), on every release since 4.0.0-ALPHA, even with a one-term index. The default 5 is safe for every realistic count. Index data and searches are unaffected.
Mechanism, terms and numbers623 words
DirectSpellChecker.suggestSimilar sizes its candidate search with int inspections = numSug * maxInspections; (DirectSpellChecker.java:359 on main 645d4fe14a20), and setMaxInspections stores any int (:145-147). With setMaxInspections(Integer.MAX_VALUE), the largest accepted value (no upper bound is documented), the product is -numSug for every even numSug and 2^31 - numSug for odd ones. A negative budget makes the one-edit pass treat its empty queue as full and dereference PriorityQueue.peek() == null (NullPointerException at :435) when the word has a one-edit neighbour, and makes the two-edit pass never run (:363, 0 < -2 is false) when it has none, so the call silently returns an empty array. On a one-word index the same settings give [lucene] for numSug 1 and 3 and [] for numSug 2. 2^30 and Integer.MAX_VALUE - 1 wrap too; at the default 5 only absurd counts do, from numSug 429,496,730 on (numSug 858,993,460 returns 4 of 10 suggestions). Every release that has the class is affected, from 4.0.0-ALPHA (2012) to 10.6.0 (run as its release candidate) and main. Computing the product in long and capping it at Integer.MAX_VALUE fixes it.
- numSug: the count argument of
suggestSimilar, "the maximum number of suggested words" (DirectSpellChecker.java:314). - maxInspections: "the maximum number of top-N inspections (default: 5) per suggestion" (
:141-143).setMaxInspectionsstores the value as given (:145-147);setMaxEdits, by contrast, rejects values outside 1..2 (:114-117). - Budget:
int inspections = numSug * maxInspections;(:359). It is passed as thenumSugargument of the protected per-pass method (:410-479), which keeps the best candidates in ajava.util.PriorityQueue(:426).
- The product wraps.
Integer.MAX_VALUEis2^31 - 1, sonumSug * maxInspectionsisnumSug * 2^31 - numSug; modulo 2^32 that is-numSugfor evennumSugand2^31 - numSugfor oddnumSug.2 * 2,147,483,647 = 4,294,967,294becomes-2. - Pass 1, one edit away (
:362). For each candidate,:435first skips uncompetitive hits:if (stQueue.size() >= numSug && score <= stQueue.peek().boost). WithnumSug = -2,0 >= -2is true on the empty queue,peek()isnull, and reading.boostthrowsNullPointerException. - Pass 2, two edits away (
:363):if (maxEdits > 1 && terms.size() < inspections). If pass 1 found nothing (so no NPE),0 < -2is false, pass 2 never runs, and:372-392return an empty array. - Small odd counts work. For every odd
numSugthe budget2^31 - numSugis positive, and up to 2^30 it is at leastnumSug, so the same checker gives the right answer for 1, 3, 5 suggestions and fails for 2, 4, 6. Above 2^30 the odd budget is smaller thannumSug:numSugInteger.MAX_VALUEgets a budget of 1 and returns 1 of 10 suggestions.
- One indexed word
lucene, wordluxeny(two edits away),setMaxInspections(Integer.MAX_VALUE): numSug 1 returns[lucene], numSug 2 returns[], numSug 3 returns[lucene]. WithsetMaxInspections(1000), numSug 2 returns[lucene]. - Index
lucene,lucent,lucerne, wordlucenx, numSug 2:setMaxInspections(1000)returns[lucene, lucent];Integer.MAX_VALUEthrowsNullPointerException: Cannot read field "boost" because the return value of "java.util.PriorityQueue.peek()" is nullatDirectSpellChecker.java:435. - Other "unlimited" values:
2^30fails for every count except 1, 5, 9, ... (2 gives-2^31, 3 gives-2^30, 4 gives 0; 5 wraps to2^30itself);Integer.MAX_VALUE - 1gives-2 * numSugfor even counts. At the default 5, numSug 429,496,730 to 858,993,459 give a negative budget, and numSug 858,993,460 gives a budget of 4: 4 of 10 suggestions, no error. Larger counts keep alternating between negative and wrapped positive budgets (858,993,461 gives 9).
Exposure and likelihood247 words
- Who is exposed. Code that calls
setMaxInspectionswith a sentinel-sized value meaning "inspect everything" and requests an even number of suggestions. With2^30every count except 1, 5, 9, ... fails (n * 2^30mod 2^32 is(n mod 4) * 2^30). - Opt-in. It needs a non-default
setMaxInspectionsvalue; nothing else non-default is needed: default codec, defaultIndexWriterConfig, one indexed term suffices, single thread, deterministic (the empty leg uses the default two-edit pass,maxEdits2). - Effect. A word for which the one-edit pass finds a candidate gets a
NullPointerExceptionfromsuggestSimilar. A word whose only neighbours are two edits away gets an empty list with no error, which looks exactly like "no suggestion exists". How often each case occurs in real traffic was not measured. Because small odd counts work, a developer who tests with 1, 3 or 5 suggestions sees nothing wrong. Nothing persists; the index and searches are not affected. - Default setting. With maxInspections 5 only counts of 429,496,730 or more wrap, far beyond any real request; the javadoc itself says "typically a low value for numSug will work very well" (
:310-311). - How likely: rare. No report of the NPE or the wrong results from this trigger since 4.0 (2012); the only public trace of the overflow is a downstream source comment (see Prior reports). The author's own example value is 5. Downstream servers that forward an inspection limit are covered in the note below; their reach was read, not executed.
The test shows that DirectSpellChecker with setMaxInspections(Integer.MAX_VALUE) returns no suggestions for luxeny against an index holding only lucene when asked for 2 suggestions, because numSug * maxInspections wraps to -2.
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.lucene.search.spell;
import java.util.Arrays;
import java.util.List;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.document.StringField;
import org.apache.lucene.index.DirectoryReader;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.index.Term;
import org.apache.lucene.store.ByteBuffersDirectory;
import org.apache.lucene.store.Directory;
import org.apache.lucene.tests.util.LuceneTestCase;
public class TestDirectSpellCheckerMaxInspections extends LuceneTestCase {
public void testMaxInspectionsMaxValue() throws Exception {
try (Directory dir = new ByteBuffersDirectory();
IndexWriter writer = new IndexWriter(dir, new IndexWriterConfig())) {
Document doc = new Document();
doc.add(new StringField("f", "lucene", Field.Store.NO));
writer.addDocument(doc);
try (DirectoryReader reader = DirectoryReader.open(writer)) {
DirectSpellChecker checker = new DirectSpellChecker();
checker.setMaxInspections(Integer.MAX_VALUE);
SuggestWord[] words = checker.suggestSimilar(new Term("f", "luxeny"), 2, reader);
// expected [lucene] (the default maxInspections finds it); actual [] (2 * MAX_VALUE wraps)
assertEquals(List.of("lucene"), Arrays.stream(words).map(w -> w.string).toList());
}
}
}
}
# in a copy of this finding's repro folder (repro/LFV-986/, which holds src/)
# Lucene checkout at the tested commit, built with: ./gradlew --no-daemon assemble (JDK 25)
LUCENE=/path/to/lucene; V=12.0.0-SNAPSHOT
CP=$LUCENE/lucene/core/build/libs/lucene-core-$V.jar
CP=$CP:$LUCENE/lucene/codecs/build/libs/lucene-codecs-$V.jar
CP=$CP:$LUCENE/lucene/test-framework/build/libs/lucene-test-framework-$V.jar
CP=$CP:$LUCENE/lucene/analysis/common/build/libs/lucene-analysis-common-$V.jar
CP=$CP:$LUCENE/lucene/suggest/build/libs/lucene-suggest-$V.jar
CP=$CP:randomizedtesting-runner-2.9.2.jar:junit-4.13.2.jar:hamcrest-3.0.jar # test-framework dependencies (Maven Central)
CP=$CP:junit-platform-console-standalone-6.1.3.jar # JUnit Platform console launcher (Maven Central)
javac -d classes -cp "$CP" $(find src -name '*.java')
# run 1: assertions on, seed DEADBEEF
java -ea -esa --add-modules jdk.incubator.vector --enable-native-access=ALL-UNNAMED -Dtests.seed=DEADBEEF \
-cp "classes:$CP" org.junit.platform.console.ConsoleLauncher \
execute --select-class org.apache.lucene.search.spell.TestDirectSpellCheckerMaxInspections
- Expected vs actual
- Expected
[lucene](what the default maxInspections returns); actual:[], with-eaand-da.
More files, environment, full output34 words
| lucene | Lucene main e1faeba08bde (2026-11-02) |
|---|---|
| jdk | 25.0.3+9-LTS |
# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
=> java.lang.AssertionError: expected:<[lucene]> but was:<[]>
org.junit.Assert.fail(Assert.java:89)
org.junit.Assert.failNotEquals(Assert.java:835)
org.junit.Assert.assertEquals(Assert.java:120)
org.junit.Assert.assertEquals(Assert.java:146)
org.apache.lucene.search.spell.TestDirectSpellCheckerMaxInspections.testMaxInspectionsMaxValue(TestDirectSpellCheckerMaxInspections.java:45)- No bound is documented: raising
maxInspections"can improve the accuracy of results", andnumSugis "the maximum number of suggested words". - Same settings, one-word index: numSug 1 and 3 return
[lucene], numSug 2 returns[]. A limit that flips with parity is an int wrap, not a design. - The class itself uses
Integer.MAX_VALUEas "no limit" formaxQueryLength.
The full argument263 words
- Contract.
setMaxInspections(DirectSpellChecker.java:141-143): "Set the maximum number of top-N inspections (default: 5) per suggestion. Increasing this number can improve the accuracy of results, at the cost of performance." No upper bound and no validation (:145-147).suggestSimilar(:314,:318): "@param numSug the maximum number of suggested words", "@return sorted list of the suggested words according to the comparator". - The class tells users to raise it.
setDistance(:281-284): "If you use a different metric than the default, you might want to consider increasing {@link #setMaxInspections(int)} to draw more candidates for your metric to rank." Integer.MAX_VALUEmeans "no limit" in this class.maxQueryLengthdefaults to it (:82; LUCENE-9102, #10144).- The parity oracle needs no promise about maxInspections. It compares two counts on the same settings: asking for up to 2 suggestions cannot rightly lose the one that a request for 1 finds.
- History. The product, the setter, the pass-2 guard and the
peek()check all come from the commit that created the class (5190ea52329, LUCENE-2507, 2010); no later commit changed the product or the guard. No test, CHANGES or MIGRATE entry boundsmaxInspections, and no test callssetMaxInspections. - Precedent. LUCENE-2119 (#3195, by mikemccand, resolved 2010-06-01):
Integer.MAX_VALUEas the hit count ofsearch(Query, int)wrapped in int and threw. The report itself calls it "a nonsense value to pass in" and says "This trips users up fairly often"; it was fixed rather than rejected (the priority queue no longer adds 1 toInteger.MAX_VALUE, andsearchcaps the count atmaxDoc()).
The devil's-advocate check170 words
The strongest case for "works as designed" (the intended-behaviour checker and the reviews by another model family): the class is @lucene.experimental (:48); its author described the parameter as "maxInspections (example: 5), the existing spellchecker uses a hardcoded 10 here. A lower value can work well here" (rmuir on LUCENE-2507 / #3581, 2010-10-01); "can improve" is no promise that results improve monotonically; and open #9195 already asks for parameter checks, so a maintainer could answer "reject silly values".
Why it fails: experimental status does not license wrong results; the author gave an example, not a limit, and no doc, test or check sets one; the parity oracle compares two counts on the same settings and needs no monotonicity in maxInspections; range checks do not reject two positive values, the default-setting leg overflows with maxInspections = 5, and rejecting Integer.MAX_VALUE would break the odd counts that work today. What remains is an impact and likelihood argument (sentinel values only), which the Low priority already reflects. Verdict: a defect.
Recommended Saturate the product at Integer.MAX_VALUE (validated)
Compute the budget in long and cap it: int inspections = (int) Math.min((long) numSug * maxInspections, Integer.MAX_VALUE); (one line in suggestSimilar). Nothing changes when the product fits; the new test and the whole suggest module pass with assertions on and off.
// numSug * maxInspections can overflow int (e.g. maxInspections = Integer.MAX_VALUE)
int inspections = (int) Math.min((long) numSug * maxInspections, Integer.MAX_VALUE);All options and trade-offs5 options · 490 words
One fix was built and validated with gradle; the alternatives were not built.
DirectSpellChecker.java:359, one file, +2/-1:
// numSug * maxInspections can overflow int (e.g. maxInspections = Integer.MAX_VALUE)
int inspections = (int) Math.min((long) numSug * maxInspections, Integer.MAX_VALUE);Trade-offs
- Behaviour is unchanged whenever the product fits in an int (a Z3 proof over positive inputs, in the model's candidate-count abstraction: the value equals today's when there is no overflow, is >= numSug and >= 1 otherwise, and the
longproduct cannot overflow). - When it does not fit, the budget is
Integer.MAX_VALUE, close to the2^31 - numSugthat small odd counts already get today: the per-pass method then keeps every candidate within the edit distance until its queue holdsInteger.MAX_VALUEentries, and pass 2 runs.java.util.PriorityQueueis not pre-sized, so nothing large is allocated up front, but the queue grows with every candidate kept: on a large index with many terms within two edits the call can do much more work and hold much more memory than today's wrong answer. Not measured. - Not covered: non-positive
numSugormaxInspections, the separate, open #9195 / SOLR-11858 request for parameter checks, which is complementary. The fix leaves them as they are today: the:435NPE when the one-edit pass finds a candidate, an empty array when it finds none, or aNegativeArraySizeExceptionat:388for a negativenumSug(executed on main and with the fix: the same outcomes).
What the checks validated
- Scratch worktree of main
645d4fe14a20, gradle: the new test fails on the stock tree and passes with the fix, with assertions on and off. The wholelucene/suggestmodule passes with assertions on and off: 32 suites (31 upstream plus the new test), 237 tests, 7 skipped, 0 failures;search.spell9 suites, 32 tests, includingTestDirectSpellChecker6 of 6.:lucene:suggest:check -x test(google-java-format, ecjLint, forbiddenApis) passes. - Standalone runner with the fixed class: the new test, the finder's two tests (the NPE and the empty leg) and the independent reproducer's three tests (including the default-5 leg with numSug 858,993,460) pass. The same one-line fix makes 4.0.0-ALPHA pass.
Reject numSug < 1 and maxInspections < 1 with IllegalArgumentException.
Trade-offs
Does not fix this bug: both values are positive here. Worth adding alongside the saturation. Not built.
Throw ArithmeticException on overflow.
Trade-offs
Loud instead of silent, but it rejects the natural "inspect everything" setting that works for odd counts today. Not built.
Clamp maxInspections to some maximum.
Trade-offs
Any cap is arbitrary, and a huge count at the default setting still overflows. Not built.
Change the protected suggestSimilar(Term, int numSug, IndexReader, int, int, float, CharsRefBuilder).
Trade-offs
The method is protected and overridable, so the signature change breaks subclasses. Not recommended. (The open draft #16847, long doc ids, does change another parameter of this method, int docfreq to long.)
Search servers that pass a request- or configuration-level inspection limit to setMaxInspections (Elasticsearch and OpenSearch take max_inspections from the request and reject only negative values; Solr passes its setting through) can inherit the error or the empty list. Elasticsearch now first reserves breaker memory for the capped product (#154837), which may stop such a value. Read, not executed.
A numeric-limits probe on the stock jars showed the NullPointerException with setMaxInspections(Integer.MAX_VALUE) and 2 suggestions. A Z3 bit-vector model of the budget then predicted the silent empty leg, and a model-based test on the stock jar confirmed every outcome.
The method in full158 words
Method: non-formal first signal, semi-formal for the empty leg (the hunt's numeric-limits hunt unit, 2026-10-28, main a9779ef3c193).
- Lead. A planning probe that tries extreme values of numeric parameters on the stock jars failed 7 of 11 checks, including the NPE for
setMaxInspections(Integer.MAX_VALUE)with numSug 2 and the truncation to 4 of 10 at the default with numSug 858,993,460. - Z3 model. The finder modelled the budget arithmetic and the result size as 32-bit bit-vectors (18 checks, 0 unexpected). T1 proves the budget is <= 0 exactly when numSug is even (with
Integer.MAX_VALUE); witness W2, found while modelling the pass-2 guard at:363, is the silently empty leg the probe did not have; F1-F4 prove the saturating fix. - Model-based test. 200 random indexes, numSug 1 to 8 and 11 maxInspections values on the stock jar: 13,950 of 44,720 checks fail, none without int overflow, and the model predicts all 98,384 observed outcomes. With the fix, 0 fail.
No Lucene report found (searched 2026-10-29: four tool runs of 51 queries, 30 manual GitHub searches, 7 JIRA queries, the files of all 414 open Lucene PRs). Found later: Elasticsearch PR #154837 (2026-07-31) added a source comment naming the int product, not a Lucene bug; novelty is open (registry: new). Closest Lucene items: #9195 and SOLR-11858 (numSug 0).
Searches6 searches
Downstream, found by a fact check after the searches: Elasticsearch PR #154837 ("Trip request breaker on phrase and term suggesters", by pmpailis, merged 2026-07-31). Its diff adds to Elasticsearch's own TermSuggester.java: "Lucene multiplies these values using int arithmetic. Compute in long and saturate so an overflowing configuration still receives the largest possible preflight estimate." That names this bug's root cause, the int product of the shard size and max_inspections, but treats an overflow as a very large queue; it does not mention a negative budget, the NPE or the empty and truncated results, does not call it a Lucene bug, and was not sent to Lucene. Its title, body, comments and reviews do not mention overflow, which is why the keyword searches missed it. Under the hunt's novelty rule a partial overlap counts as "known" when in doubt, and the registry has done so for comparable outside source notes; the registry still says new, and the decision is open.
Related, not reports of this bug: open #9195 (LUCENE-8147, 2018, by rmuir, 0 comments, type:bug) says "if i ask for zero suggestions I think it may give a confusing NPE instead ... There are probably other cases too: we should add all the missing checks and give IllegalArgumentExceptions"; open SOLR-11858 hits the same peek() line, attributed to zero or negative numSug; SOLR-4320 and SOLR-5090 reach it with a count of 0. None names maxInspections, the product, an overflow, or the empty and truncated results, and the fixes are disjoint: range checks do not reject two positive values, and the saturating product does not cover numSug 0.
Context: #3581 (LUCENE-2507, introduced the parameter), #10144 (LUCENE-9102, Integer.MAX_VALUE as this class's "no limit"), #3195 (LUCENE-2119, the precedent). The only open PR that touches the file, #16847 (long doc ids, draft), leaves line 359 unchanged. Gaps: keyword search only; GitHub Discussions and chat channels were not searched.
| Where | When | How | Found |
|---|---|---|---|
| GitHub issues, PRs and commits (apache/lucene and downstream) | 2026-10-29 | four scripted prior-report search runs of 51 queries each (three by the prior-report search: overflow, NPE, downstream wording; one earlier by the finder); 30 manual searches over apache/lucene, apache/solr, apache/lucene-solr, elastic/elasticsearch, opensearch-project/OpenSearch and apache/lucenenet | no report; #9195 related (same NPE line, numSug 0) |
| Open PRs | 2026-10-29 | the changed files of all 414 open apache/lucene PRs | only #16847 touches DirectSpellChecker.java, not line 359 |
| JIRA archive | 2026-10-29 | 7 JQL queries over LUCENE and SOLR; 11 issues read in full | SOLR-11858, SOLR-4320, SOLR-5090: the same NPE line with a count of 0 |
| Mailing lists and web | 2026-10-29 | 25 queries over the Solr and Lucene user, dev and issues lists; 24 web queries | no report |
| GitHub (recheck) | 2026-10-29 | the 35 apache/lucene items updated since 2026-10-28 and the files of their 27 PRs; 8 searches | nothing new |
| Downstream source (fact check) | 2026-10-29 | read-only GitHub API reads of the Elasticsearch, OpenSearch, Solr and Lucene.NET suggester sources, and of PR #154837 (metadata, files, patch, comments, reviews) | Elasticsearch #154837: a source comment names the int product; no Lucene bug or symptom reported |
As old as the class: introduced in 2010 by 5190ea52329 (LUCENE-2507), first released in 4.0.0-ALPHA (2012). All 132 releases from 4.0.0-ALPHA to 10.5.2, 10.6.0 (run as its release candidate) and main fail the same way; no release with the class is clean. Not fixed upstream.
Versions tested6 rows
Introduced by 5190ea52329, "LUCENE-2507: Add automaton spellchecker" (Robert Muir, @rmuir, 2010-10-01; JIRA LUCENE-2507, migrated as #3581; Subversion era, no PR). That commit already had the unvalidated setter, the int product, the pass-2 guard and the unchecked peek(); LUCENE-7439 (faf3bc3134c, 2016) only renamed boost to score on the peek() line. A bytecode check of all 134 suggest jars (132 releases, the RC, main) confirms the int multiply and the peek() line. Not fixed: the newest commit on the file is de3b294be44 (formatting, 2023-10-10), and main 645d4fe14a20 still has the line at :359.
| Version or commit | Verdict | Result |
|---|---|---|
main 645d4fe14a20 | affected | the brief's test fails (gradle and the standalone runner, assertions on and off; 10 of 10 iterations) |
main a9779ef3c193, 10.6 RC 4ae946e5a9b9 | affected | NPE at DirectSpellChecker.java:435, [] for the two-edit word, parity row, 4 of 10 at the default |
| 10.0.0 to 10.5.2 (12 releases) | affected | the same four outcomes; the brief's test fails on 10.5.2 (assertions off) |
| 4.0.0-ALPHA to 9.12.3 (120 releases) | affected | the same four outcomes (release probe, Maven Central jars, SHA-1 checked) |
| 1.x to 3.x | class absent | no DirectSpellChecker (jar listings of 3.1.0 and 3.6.2; every 3.x tag) |
| main + fix; 4.0.0-ALPHA + fix | not affected | the brief's test passes; the whole lucene/suggest module, 237 tests, 0 failures; 4.0.0-ALPHA passes the release probe |
An independent reproducer, an intended-behaviour devil's advocate, an adjudication and a credibility audit re-ran the repros and agree; reviews by another model family (OpenAI GPT-5.6) could only read code and agree on the root cause. The brief re-ran everything on main 645d4fe14a20 and validated the fix with gradle.
Checks and verdicts426 words
The cross-model reviews were reading-only: they could not run Java. The refute-lens review returned a procedural "not real, medium" while agreeing on the root cause; its notes ask for a re-run before treating it as a rejection, and a separate re-run failed 2 of 2. No executing cross-model review exists.
| Check | Verdict | What it found |
|---|---|---|
| finder | candidate | 2-test JUnit fails 2 of 2 on main (-ea, -da); its main() probe fails on main, the 10.6 RC, 10.5.2, 9.12.3, 9.0.0 and 8.0.0; Z3 model and model-based test as above; with the fix 0 fail and TestDirectSpellChecker passes 6 of 6. |
| cross-model review, refute lens | not real, medium (procedural, reading-only) | OpenAI GPT-5.6; root cause agrees, inputs in contract, intended argument weak. Its point that "can improve" is no monotonicity promise made the finder compare with the exhaustive answer (results unchanged). |
| Independent reproducer | real | Own test in a foreign package with other words fails 3 of 3 on main (-ea, -da), 10.5.2 and 9.12.3, controls passing first; a one-line fix makes it pass. |
| Cross-model review, intended lens | real, low (reading-only) | OpenAI GPT-5.6; the works-as-designed case is narrow. The cross-model review step re-ran the repro (fail) and with the fix (pass). |
| Intended-behaviour devil's advocate | real | Built the strongest intended case and rejected it; its parity probe fails 2 of 4 on main and 10.5.2 with both odd-count controls passing, and passes 4 of 4 with the fix. |
| adjudication | confirmed, high | Re-ran the independent reproduction's test on main: 3 of 3 fail (-ea, -da); novelty new with the #9195 overlap disclosed. |
| credibility audit | keep | Own main() program reproduces all four outcomes (NPE, empty, parity, 4 of 10 at the default). |
| this doc's runs | reproduced, fix validated | Gradle and standalone runs on 645d4fe14a20 and 10.5.2; the finder's and the independent reproduction's tests re-run on 645d4fe14a20 (fail) and with the fix (pass); the whole suggest module passes with the fix. |
| Fact checks of this doc and diagram check (2026-10-29) | bug, mechanism and history hold; wording corrected; novelty open | An independent fact check re-ran the repro with gradle and its own probe (1 error, fixed). A second, reading-only fact check by another model family (OpenAI GPT-5.6) found 5 errors: 3 fixed, 1 rejected (the run count was right), and the downstream comment of Elasticsearch #154837, now disclosed in Prior reports, with the novelty decision open. The diagram check found no errors; three on-screen wordings were updated and the diagram re-rendered. |
The diagram as a still image, generated with AI and checked against the code. It is under 10 MB, the limit for a GitHub attachment.
- PNG 481 KB
Finding record, Lucene and upstream sources, repro files and diagram notes1,237 words
- Registry title
- DirectSpellChecker.suggestSimilar computes its candidate budget numSug * maxInspections in int: with setMaxInspections(Integer.MAX_VALUE) ('inspect everything') and any even numSug the budget wraps to -numSug, so the call throws NullPointerException or silently returns no suggestions
- Area
- suggest / spell checking (DirectSpellChecker.suggestSimilar candidate budget numSug * maxInspections)
- Priority
- Low, rank 435: An Integer.MAX_VALUE maxInspections with an even numSug makes the widely used DirectSpellChecker throw an NPE or silently return nothing for edit-distance-2 words, a deterministic suggestion bug on an opt-in boundary setting.
- Impact class
- silent wrong results
- Impact note
- Applications on every release from 4.0.0-ALPHA to 10.5.2, 10.6.0 (run as its RC) and main that configure DirectSpellChecker with a sentinel-sized maxInspections (Integer.MAX_VALUE as 'inspect everything', 2^30, MAX-1 and similar: an opt-in setter whose default is 5) and ask for an even number of suggestions get a NullPointerException from suggestSimilar when the word has an edit-distance-1 neighbour, or silently an empty suggestion array when its only neighbours are at edit distance 2, at any index size (10-term indexes suffice); odd numSug and ordinary values work, and with the default maxInspections only an absurd numSug (429,496,730 or more) wraps, e.g. numSug 858,993,460 silently returns 4 of 10 suggestions; no index data or search hits are affected.
- Summary
- You tell DirectSpellChecker to inspect as many candidates as it likes with setMaxInspections(Integer.MAX_VALUE) and ask suggestSimilar for 2 suggestions; you expect the best suggestions, at least as good as with a budget of 1000. Instead the internal budget numSug * maxInspections overflows int to -2: if the word has a one-edit neighbour the call throws NullPointerException, and if its only neighbours are two edits away it returns an empty array with no error. The outcome flips with the parity of numSug (1 and 3 work, 2 and 4 fail), and the one-line fix is to compute the budget in long and cap it at Integer.MAX_VALUE.
- Introduced
- 5190ea52329 ('LUCENE-2507: Add automaton spellchecker', by Robert Muir (rmuir), author and commit date 2010-10-01, svn r1003642; JIRA LUCENE-2507, migrated as #3581; no pull request, Lucene used Subversion then): the commit that created DirectSpellChecker already had the unvalidated setMaxInspections, the int product 'int inspections = numSug * maxInspections;', the pass-2 guard 'terms.size() < inspections' and the unchecked stQueue.peek() dereference. The peek line was reworded by LUCENE-7439 (faf3bc3134c, 2016, boost -> score) without changing the check [read: git blame on main, git log -S on the product and the guard (only 5190ea52329), --diff-filter=A; exec: every release, see affected_versions] (2010-10-01 (author and commit date of 5190ea52329; LUCENE-2507 resolved the same day))
- First affected version
- 4.0.0-ALPHA [exec] (the first release with DirectSpellChecker, tag 2012-06-29; first GA release 4.0.0, 2012-10-11). The class is absent from every 1.x-3.x release [exec: jar listing of lucene-spellchecker 3.1.0 and 3.6.2; read: git ls-tree of every 3.x tag], so no release that has the API is unaffected
- Affected versions
- Every release that has DirectSpellChecker [exec]: all 132 Maven Central releases from 4.0.0-ALPHA to 10.5.2 (4.0.0-ALPHA, 4.0.0-BETA and 130 GA releases: 23 4.x, 15 5.x, 18 6.x, 14 7.x, 27 8.x, 21 9.x, 12 10.x; latest 9.x 9.12.3, 10.x through 10.5.2), 10.6.0 (released 2026-10-29; run as its RC, branch_10_6 4ae946e5a9b9) and main a9779ef3c193, each with the same outcomes (NPE leg, empty edit-distance-2 leg, parity row, default-setting truncation to 4 of 10); 4.0.0 also with -da. Bytecode check on all 134 suggest jars: the budget is an int multiply and the NPE frame line holds PriorityQueue.peek() then a ScoreTerm.boost read. The one-line saturating fix makes 4.0.0-ALPHA pass. Not fixed upstream: main 645d4fe14a20 still has 'int inspections = numSug * maxInspections;' at :359, and the newest commit on DirectSpellChecker.java on main, branch_11x, branch_10x and branch_10_6 is de3b294be44 (formatting, author date 2023-10-10) (recheck 2026-10-29). Caveat: all runs used JDK 25.0.3 on Linux aarch64
- Status
- documented (brief, checks and a per-finding doc exist)
- Verification
- Confirmed by AI-run checks: an independent repro, a cross-model review that could only read the code (its “not real” is procedural), an intended-behaviour check, a prior-report search and a skeptical audit; not yet triaged by a person.
- Found by
- non-formal
- Upstream and prior reports
- none found (the prior-report record verdict new, searched 2026-10-29, read-only; recheck 2026-10-29): four scripted prior-report searches (51 queries each over GitHub issues, PRs and commits, downstream repositories, the dev@, java-user@, issues@ and commits@ lists, the JIRA archive and the web; 0 high hits apart from the design origin #3581), 30 GitHub search phrasings over apache/lucene, apache/solr, apache/lucene-solr, elastic/elasticsearch, opensearch-project/OpenSearch and apache/lucenenet, the changed files of all 414 open apache/lucene PRs (only #16847, long doc ids, touches DirectSpellChecker.java, and not line 359), 7 JQL queries with 11 JIRA issues read in full, 25 mailing-list queries and 24 web queries. Related, not the same bug: open #9195 (LUCENE-8147, 2018, 'DirectSpellChecker needs better parameter checks', 0 comments) and open SOLR-11858 report the NullPointerException at the same stQueue.peek() line reached with numSug = 0 and ask for IllegalArgumentException checks ('There are probably other cases too'); SOLR-4320 and SOLR-5090 (fixed on the Solr side) reach it with a suggestion count of 0. None names maxInspections, the numSug * maxInspections product, overflow, or the silently empty or truncated results, and the fixes are disjoint (range checks do not reject two positive values; the saturating product does not cover numSug = 0). Context only: #3581 (LUCENE-2507, which introduced the parameter), #10144 (LUCENE-9102, Integer.MAX_VALUE as this class's 'no limit' default), #4424, es#34282, es#24655. Recheck: the 35 apache/lucene issues and PRs updated since 2026-10-28 name neither the class nor the parameter, and none of their 27 PRs changes a file under lucene/suggest/**/spell/; searches 'maxInspections' (#3581 only), 'setMaxInspections', 'DirectSpellChecker NullPointerException', 'DirectSpellChecker created:>=2026-10-27', 'spell is:pr created:>=2026-10-27' and 'suggestSimilar is:pr updated:>=2026-10-27' return nothing new. Tracking issue #16937 does not mention spell checking.
- Lucene commit tested
- main
645d4fe14a20(tested 2026-10-29) through gradle in a scratch worktree and through the standalone JUnit runner, with assertions on and off and 10 iterations; the 10.5.2 release jars (assertions off). Earlier hunt runs: maina9779ef3c193(DirectSpellChecker.javaidentical), the 10.6 RC4ae946e5a9b9and all 132 releases from 4.0.0-ALPHA to 10.5.2. The fix was validated in the scratch worktree: the new test, the wholelucene/suggestmodule with assertions on and off, and:lucene:suggest:check -x test.
- Finding record
lucene/suggest/src/java/org/apache/lucene/search/spell/DirectSpellChecker.java:48 (@lucene.experimental), :82 (maxQueryLength = Integer.MAX_VALUE), :140-147 (setMaxInspections javadoc, no validation), :314 and :318 (suggestSimilar @param numSug / @return), :359 (int product), :362-368 (pass-2 guard), :435 (stQueue.peek().boost)
- The finding in 30 seconds
lucene/suggest/src/java/org/apache/lucene/search/spell/DirectSpellChecker.java:67, (at 645d4fe14a20) :114-117, :141-147, :314, :358-363, :372-392, :410-479, :426, :435
- Who is affected
lucene/suggest/src/java/org/apache/lucene/search/spell/DirectSpellChecker.java:310-311 (at 645d4fe14a20)
- Why it looks unintended
lucene/suggest/src/java/org/apache/lucene/search/spell/DirectSpellChecker.java:48, (at 645d4fe14a20) :82, :141-147, :281-284, :314, :318github.com/apache/lucene/issues/3195 (LUCENE-2119 body, read 2026-10-29)github.com/apache/lucene/issues/3581 (LUCENE-2507 comment of 2010-10-01 on maxInspections, read 2026-10-29)
- Suggested fix
lucene/suggest/src/java/org/apache/lucene/search/spell/DirectSpellChecker.java:410-479 (at 645d4fe14a20) (protected per-pass method), :426 (unsized PriorityQueue), :435
- Prior reports
github.com/apache/lucene/issues/9195 (body, read 2026-10-29)
- TestDirectSpellCheckerMaxInspections.java (JUnit test):
repro/LFV-986/src/org/apache/lucene/search/spell/TestDirectSpellCheckerMaxInspections.java(49 lines)
It is made in parallel with this doc; every code row and executed value on screen is checked against the code and the diagram's own probe (the part 7 note on non-positive inputs against the fact-check resolution's probe). Three captions and notes were corrected after the fact checks and the diagram was re-rendered.
assets/LFV-986/bug.png(481 KB)
![Summary card for DirectSpellChecker with setMaxInspections(Integer.MAX_VALUE) and numSug 2: the budget numSug * maxInspections = 2 * 2147483647 wraps to -2 (numSug 1, 2, 3, 4 give 2147483647, -2, 2147483645, -4). Index lucene, lucent, lucerne, word lucenx: expected [lucene, lucent], actual NullPointerException at DirectSpellChecker.java:435. Index lucene, word luxeny: expected [lucene], actual [] with no error, because pass 2 never runs (DirectSpellChecker.java:363). Controls: odd numSug 1 and 3 still get [lucene] for luxeny, and the default maxInspections 5 gives both expected answers. Fix: compute the product in long and cap it at Integer.MAX_VALUE; every release from 4.0.0-ALPHA to 10.5.2 and main is affected.](../assets/LFV-986/bug.png)