Flexible StandardQueryParser throws NumberFormatException on a fractional phrase slop such as "a b"~30.5

The flexible StandardQueryParser documents proximity search as "test failure"~4 and declares QueryNodeException as the failure of parse(). Its lexer has one NUMBER token for integers and decimals, so 30.5 after ~ is a legal token, but since LUCENE-10223 (9.1.0) the phrase rule converts it with Integer.parseInt. parse("\"a b\"~30.5", "f") therefore throws java.lang.NumberFormatException: For input string: "30.5", which parse() does not convert, while Lucene 8.11.4, 9.0.0 and today's classic QueryParser return PhraseQuery "a b"~30.

The same helper serves the @N operator and the integer arguments of the fn: interval functions, and fails the same way for a fraction or for an integer above Integer.MAX_VALUE. Nothing is searched and no result is wrong: a query string is refused with an undeclared exception type. A one-helper grammar fix (throw the declared ParseException) is validated with the whole queryparser test suite and check; restoring the 9.0.0 truncation for the slop is validated as an alternative.

1. Lexer. <NUMBER: (<_NUM_CHAR>)+ ( "." (<_NUM_CHAR>)+ )? > (StandardSyntaxParser.jj:160): one token type for integers and decimals, shared by the phrase slop, the fuzzy similarity (term~0.7), boosts, @N and the fn: arguments. "a b"~30.5 lexes as <QUOTED> <TILDE> <NUMBER>.
2. Phrase rule. QuotedTerm reads ( <TILDE> slop = <NUMBER> { q = new SlopQueryNode(q, parseInt(slop)); } )? (.jj:833; generated StandardSyntaxParser.java:1293).
3. Helper. public static int parseInt(Token token) { return Integer.parseInt(token.image); } (.jj:119-121; generated :109-111). Integer.parseInt("30.5") throws NumberFormatException, and so does Integer.parseInt("3000000000").
4. Entry point. parse() (.jj:99-113) passes a ParseException on (:103; it is a QueryNodeParseException) and wraps an Error into one (:106). NumberFormatException is a RuntimeException, so it passes through parse(), QueryParserHelper.parse and StandardQueryParser.parse unchanged.

parseInt has 12 calls at 10 places, all on NUMBER tokens: the slop (.jj:833), @N (:361), and fn:atLeast, fn:maxwidth, fn:maxgaps, fn:within, fn:extend (two), fn:notWithin, fn:wildcard and fn:fuzzyTerm (two). The study ran 10 such inputs on main 7cecf79523, for example (a b)@1.5, fn:maxwidth(2.5 a) and "a b"~3000000000 (the slop site with a value too large for an int): all 10 throw NumberFormatException. The fact check re-ran all 12 calls on 645d4fe14a20 (13 inputs): each throws.

In the same grammar, FuzzyOp reads its NUMBER with parseFloat and checks it, throwing ParseException for a bad value (.jj:712-728), so term~1.1 is a clean QueryNodeException. "a b"~30x and "a b"~-1 are ParseExceptions too. Only the parseInt sites let a lexed number fail with an unchecked exception.

• Who is hit. Applications that pass query strings to the flexible StandardQueryParser. Default parser settings, any analyzer, no index needed. The classic QueryParser is not affected.
• What goes wrong. Code that catches QueryNodeException to answer "invalid query" gets an unexpected NumberFormatException instead. What happens next depends on the application; one that catches nothing else answers that request with a generic server error.
• When. A number after ~ on a phrase with a fraction or above Integer.MAX_VALUE, and decimals in @N and in the integer arguments of fn: functions. Decimals after ~ are normal on terms (term~0.7, the old fuzzy-similarity form, still accepted), so the same habit on a phrase hits this. Users who relied on the 9.0.0 truncation got the exception on upgrade.
• Limits. Deterministic and immediate. No wrong results, no hang, no resource growth, no index effect. The registry rates the input rare, and no public report exists in the four and a half years since 9.1.0 (released 2022-03-22).

The test sits in the parser's test package, as Lucene's own parser tests do, but uses no package-private member. Nothing in it is randomized; seeds DEADBEEF and 5EED1234 give the same failure.

As a regression test it is the same test, except that ecjLintTest rejects the unused catch parameter e and accepts it written as catch (@SuppressWarnings("unused") QueryNodeException e), as QueryParserBase.java:803-805 writes it. With that change it fails on stock, passes with the recommended fix and passes :lucene:queryparser:check -x test.

The other shapes of the same root cause ("a b"~3000000000, (a b)@1.5, the fn: arguments) are left out of the test.

TestFractionalPhraseSlop > testFractionalPhraseSlop FAILED
    java.lang.AssertionError: expected a Query or a QueryNodeException, got java.lang.NumberFormatException: For input string: "30.5"
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:62F18F6838A133AC]:0)
        at org.apache.lucene.queryparser.flexible.standard.TestFractionalPhraseSlop.testFractionalPhraseSlop(TestFractionalPhraseSlop.java:32)

        Caused by:
        java.lang.NumberFormatException: For input string: "30.5"
            at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:67)
            at java.base/java.lang.Integer.parseInt(Integer.java:565)
            at java.base/java.lang.Integer.parseInt(Integer.java:662)
            at org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser.parseInt(StandardSyntaxParser.java:110)
            at org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser.QuotedTerm(StandardSyntaxParser.java:1293)
            at org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser.Term(StandardSyntaxParser.java:1256)
            ...
            at org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser.parse(StandardSyntaxParser.java:91)
            at org.apache.lucene.queryparser.flexible.core.QueryParserHelper.parse(QueryParserHelper.java:214)
            at org.apache.lucene.queryparser.flexible.standard.StandardQueryParser.parse(StandardQueryParser.java:279)
            at org.apache.lucene.queryparser.flexible.standard.TestFractionalPhraseSlop.testFractionalPhraseSlop(TestFractionalPhraseSlop.java:28)
:lucene:queryparser:test (FAILURE): 1 test, 1  failure

# the same message on 10.6.0 (StandardSyntaxParser.java:111 <- :1294), 10.5.2 and 9.1.0 (:110 <- :1063), assertions on and off

• The declared failure. StandardQueryParser#parse and QueryParserHelper#parse: "@throws QueryNodeException if something wrong happens along the three phases" (StandardQueryParser.java:274, QueryParserHelper.java:211). StandardSyntaxParser#parse: "@throws ParseException if the parsing fails" (.jj:96). SyntaxParser#parse declares only QueryNodeParseException (SyntaxParser.java:29). ParseException extends QueryNodeParseException, which extends QueryNodeException.
• Other bad numbers are rejected cleanly. "a b"~30x and "a b"~-1 give ParseException; FuzzyOp throws ParseException for term~1.1, and TestQPHelper.java:560 asserts it: assertQueryNodeException("term~1.1"); // value > 1, throws exception. testException (:1002-1010) asserts QueryNodeException for malformed text such as "\"some phrase".
• A regression that no note announced. 9.0.0 converted the slop with (int) Float.parseFloat(slop.image) (.jj:445 at releases/lucene/9.0.0) and returns slop 30; before LUCENE-9528 the conversion sat in catch (Exception ignored). f725b27e12a (LUCENE-10223, PR #429) swapped in parseInt(slop) in the same diff that added the parseInt/parseFloat helpers for the interval functions. The PR has no description; its only review comments are a static-analysis bot's findings in two new node classes and the author's reply telling the bot to ignore one. Issue #11259 never mentions slop. The 9.1 migration note lists only fn: and @ as parsing differently (MIGRATE.md:1116-1122).
• The classic parser, whose syntax the flexible one "borrows most of" (StandardQueryParser.java:63-65), accepts the input as slop 30 (QueryParserBase.java:798-807).
• A contrast in another parser. #12251 / PR #12277 (9.9.0) first proposed wrapping the NumberFormatException that RegExp's Integer.parseInt throws for an oversized repeat count. That part was dropped because, as rmuir commented, "NumberFormatException is a subclass of IllegalArgumentException already", and IllegalArgumentException is what RegExp declares (RegExp.java:510-512). The merged change only adds a clean IllegalArgumentException for an out-of-order repetition range (mikemccand: "we catch an invalid usage and throw a clean exception"); RegExp.java:1274 and :1279 still call Integer.parseInt unwrapped. So the raw NumberFormatException was left alone there because it already is the declared type. Here the declared exception is the checked QueryNodeException, and NumberFormatException is not a subtype of it.

The strongest case (rated moderate by the cross-model reviewer, and raised by the prior-report search): slop is an int everywhere (SlopQueryNode, PhraseQuery), the docs show only integer slops, so "a b"~30.5 is outside the documented syntax and any exception is a fair answer; @throws QueryNodeException does not promise that no RuntimeException escapes; and the XML query parser even has a test asserting that NumberFormatException escapes for an empty slop attribute (TestCoreParser.java:163-167, open issue #12708 is about that test).

Why it fails: the claim does not need the input to be accepted, and the test passes on a QueryNodeException too. For a text parser, "outside the syntax" means a syntax error, and this grammar routes syntax errors, even Error, to QueryNodeParseException and already rejects term~1.1 and "a b"~30x that way. 9.0.0 accepted the input and no migration note announced the change. The XML parser takes an XML document (CoreParser.parse(InputStream), CoreParser.java:141), not query-syntax text, and declares its own ParserException; its test pins what one malformed attribute throws today rather than stating a rule, so it is a weak precedent for a text grammar. A maintainer may still prefer rejection to truncation: that is the recommended fix.

Both fixes go into the grammar; the generated parser is then rebuilt with ./gradlew :lucene:queryparser:regenerateFlexibleStandardSyntaxParser, which formats it and updates the checksum file. Both were validated in a scratch worktree of 645d4fe14a20 (OpenJDK 25.0.3, seed DEADBEEF), removed afterwards. The choice between them is the maintainers': reject a fractional slop, or accept it as 9.0.0 did.

In StandardSyntaxParser.jj (:119-121). After regeneration the change is 3 files, +101/-90: the grammar, the generated parser (most of its lines are JavaCC renaming its lookahead methods, whose names carry .jj line numbers) and the checksum file. Every NUMBER that is not an int (a fraction, or more than Integer.MAX_VALUE) becomes the declared ParseException; integers parse as today. For all 12 calls this follows from the shared helper; the shadow probe below ran three of the inputs.

-  public static int parseInt(Token token) {
-    return Integer.parseInt(token.image);
+  public static int parseInt(Token token) throws ParseException {
+    try {
+      return Integer.parseInt(token.image);
+    } catch (NumberFormatException e) {
+      throw new ParseException(
+          new MessageImpl(QueryParserMessages.INVALID_SYNTAX, "expected an integer: " + token.image));
+    }
   }

Trade-offs

• "a b"~30.5, accepted by 9.0.0, is now rejected; since 9.1.0 it has failed anyway, only with the wrong type.
• parseInt is public static, so the added checked throws ParseException changes its signature for any outside caller; within Lucene only the generated parser calls it. A new private helper would avoid that.
• The visible message is the generic Syntax Error, cannot parse "a b"~30.5: , the same as for "a b"~30x: parse() calls setQuery on every ParseException (.jj:104), and setQuery replaces the message (QueryNodeParseException.java:55-58). The "expected an integer" text is therefore lost; a plain new ParseException() behaves the same. Keeping details is the older open #7934.
• Could add a CHANGES entry.

What the checks validated

• Stock: the repro fails with assertions on and off. Fix: JavaCC regeneration succeeds; the repro passes with assertions on and off.
• The whole :lucene:queryparser:test: 534 tests (the module's own plus the repro), 1 skipped, 0 failures.
• With the lint-clean regression test in place, :lucene:queryparser:check -x test passes: the three parser-checksum checks, checkGoogleJavaFormat, ecjLintMain, ecjLintTest, forbiddenApisMain, forbiddenApisTest, checkLicenses.
• Shadowed onto the 645d4fe14a20 jars, the regenerated parser turns "a b"~30.5, "a b"~3000000000 and (a b)@1.5 into ParseException; "a b"~30 still gives slop 30.

In QuotedTerm (.jj:833); after regeneration 3 files, +4/-4. "a b"~30.5 gives slop 30 and "a b"~3000000000 slop 2147483647, as in 9.0.0 and the classic parser.

-  ( <TILDE> slop = <NUMBER> { q = new SlopQueryNode(q, parseInt(slop)); } )?
+  ( <TILDE> slop = <NUMBER> { q = new SlopQueryNode(q, (int) Float.parseFloat(slop.image)); } )?

Trade-offs

Silent truncation (30.5 becomes 30), which is what users had before 9.1.0. It fixes only the slop: (a b)@1.5 and the fn: arguments still throw NumberFormatException. Combined with the recommended fix for the helper, it gives 9.0.0 compatibility for the slop and the declared exception elsewhere; that combination was not run.

What the checks validated

The repro passes; the whole :lucene:queryparser:test: 534 tests, 1 skipped, 0 failures. Shadowed onto the jars: slop 30 and 2147483647 as above, (a b)@1.5 still NumberFormatException. A first regeneration in the same worktree left the unchanged generated files unformatted (the formatter task is incremental); it was discarded and redone with the formatter forced to re-run.

Add a catch (NumberFormatException e) next to the Error branch (.jj:106-112) and wrap it in a QueryNodeParseException. Not run.

Trade-offs

Keeps the helper's signature and covers every call, but also hides a NumberFormatException from any other code reached during parsing, and sits further from the cause.

Not run.

Trade-offs

The lexer cannot tell a phrase slop from a fuzzy similarity (term~0.7 needs the decimal), so this needs lexical states or grammar restructuring for a small gain.

Method: non-formal baseline of the study (fuzzing).

• The fuzzer, QueryParserFuzzer, fed generated strings to the classic, flexible and simple query parsers. Its oracle: each parser may only throw its documented exception, and a parsed query must survive toString, equals, count and rewrite. It ran for 1.5 + 7 + 20 minutes, about 470,000 executions, and produced five query-parser findings; this is one of them. It was a blind find.
• An independent AI verifier (same model family) wrote its own JUnit test and a cross-version main(): the test failed on main 7cecf79523, 9.1.0 and 10.1.0; 9.0.0 accepted the input as slop 30; all 10 parseInt inputs failed. It found the introducing commit by reading the history.
• Later steps cut the repro to one test (2026-10-29 noise audit), re-ran it on 645d4fe14a20, swept the releases, and validated two fixes with gradle.