Flexible query parser prints the phrase slop through a float: "a b"~2147483647 comes back as ~2147483648, which it cannot parse

The flexible query parser reads the phrase slop of "..."~N with Integer.parseInt into SlopQueryNode, whose value is an int. To print it, toQueryString() and toString() box the int into a Float and print the float's long value. A float is exact only up to 2^24, so 2,071,986,176 of the 2,147,483,648 non-negative slops print a different number: "a b"~16777217 prints ~16777216, and the 64 slops from 2147483584 to 2147483647, including the "any distance" value Integer.MAX_VALUE, print ~2147483648, which the same parser rejects with NumberFormatException. That breaks the documented promise that a node tree prints "a query string that can be parsed back by the original text parser". The Query built from the original text is correct, so searches are unaffected. The printer has not changed since 2.9.0, and the parsed "a b"~2147483647 has printed ~2147483648 in every release. Up to 9.0.0 the grammar rounded the slop itself, so that text still read back as the same slop; since 9.1.0, when the grammar started keeping the exact int, the text is rejected and other parsed slops above 2^24 print a different value. PhraseSlopQueryNode has the same printer. The fix is Integer.toString(this.value) in both classes.

• Parse side, exact int. The grammar builds new SlopQueryNode(q, parseInt(slop)) (StandardSyntaxParser.jj:833), and parseInt is Integer.parseInt(token.image) (:119-121).
• The node is int-typed: private int value; (SlopQueryNode.java:35), the constructor SlopQueryNode(QueryNode query, int value) (:41) and int getValue() (:57-59).
• Print side, through a float (SlopQueryNode.java:61-65; PhraseSlopQueryNode.java:53-57 is identical):

private CharSequence getValueString() {
  Float f = Float.valueOf(this.value);   // int -> float: rounds above 2^24
  if (f == f.longValue()) return "" + f.longValue();
  else return "" + f;                    // never taken for an int
}

toQueryString returns the child's text + "~" + getValueString() (:73-76), and toString prints <slop value='...'> with the same helper (:68-70). The body is a copy of BoostQueryNode.getValueString (BoostQueryNode.java:80-84), where the value really is a float (:33).

• Exhaustively over every non-negative int: 2,071,986,176 of 2,147,483,648 slops print a different number, exactly the ints in [2^24, 2^31) that are not floats (2^31 - 2^24 - 7 x 2^23). The smallest is 16777217 -> 16777216; the largest error is 64.
• The 64 slops 2147483584..2147483647 print 2147483648, which Integer.parseInt rejects, so re-parsing throws NumberFormatException: For input string: "2147483648" from StandardSyntaxParser.parseInt.
• Hand-built nodes: SlopQueryNode(q, 20000001).toString() shows <slop value='20000000'>; PhraseSlopQueryNode(q, 123456789) prints ~123456792.
• Only non-negative slops are in scope: the grammar's NUMBER token has no sign (StandardSyntaxParser.jj:160) and PhraseQuery rejects a negative slop (PhraseQuery.java:173-174).

SlopQueryNodeBuilder copies getValue() into the PhraseQuery (SlopQueryNodeBuilder.java:47), so StandardQueryParser.parse("\"a b\"~2147483647", "f") searches with slop 2147483647. Inside Lucene, toQueryString output only goes into exception messages (for example QueryTreeBuilder.java:161). No processor or builder creates PhraseSlopQueryNode, so that half is reachable only through hand-built trees.

• Who. Code that turns node trees back into text to log, display, cache, forward or rewrite a query and then parses or compares it. The usual source is the syntax tree from StandardQueryParser.getSyntaxParser().parse(text, field), plus hand-built SlopQueryNode and PhraseSlopQueryNode trees. No option is needed.
• Trigger. A phrase slop above 16,777,216 that a float cannot hold. The most likely such value is the "any distance" slop "..."~2147483647, which a 2004 java-user answer recommends ("red fox"~2147483647; a 2010 java-user thread uses the same value as a SpanNear slop in the XML query parser). How often applications use it was not measured.
• Effect. For slops 2147483584..2147483647 the printed query cannot be parsed again (NumberFormatException). Other slops above 2^24 that a float cannot hold parse into a slightly different query (off by at most 64); whether that changes matching or scoring for a given document was not measured. toString() shows the same wrong value, which can mislead debugging.
• Not affected. Search results: the query is built from the exact int. In processed trees a SlopQueryNode survives only above a tokenized or multi-phrase child (PhraseSlopQueryNodeProcessor.java:43-50), which prints in internal form ([TP[...]], [MTP[...]], commented as not re-parseable), so for them only the displayed number differs. The bug concerns raw syntax-parser trees and hand-built nodes.
• Likelihood: rare. It needs both the print-and-reuse pattern and a huge slop; it is deterministic when both are present, and the recorded prior-report searches found no report.

The class sits in SlopQueryNode's package (org.apache.lucene.queryparser.flexible.core.nodes) so it can live next to the class upstream; it uses no package-private access. The assertion has no random input; LuceneTestCase still randomizes locale, timezone, codec and similarity, none of which affects the comparison, and the seed pins them. The test does not assert the NumberFormatException of re-parsing the printed text: that exception comes from the grammar's own Integer.parseInt, a separate parse-side issue.

Regression test: this same class (or its one method added to the existing TestQueryNode); it passes with the recommended fix.

cp $REPRO/TestSlopQueryNodeToQueryString.java $LUCENE/lucene/queryparser/src/test/org/apache/lucene/queryparser/flexible/core/nodes/
cd $LUCENE && ./gradlew :lucene:queryparser:test --tests TestSlopQueryNodeToQueryString -Ptests.seed=DEADBEEF -Ptests.asserts=true
# assertions off fails the same way: -Ptests.asserts=false

# gradle, stock 645d4fe14a20, -Ptests.seed=DEADBEEF -Ptests.asserts=true (stack trimmed; -Ptests.asserts=false identical)
TestSlopQueryNodeToQueryString > testMaxIntSlopRoundTrips FAILED
    org.junit.ComparisonFailure: expected:<f:"a b"~214748364[7]> but was:<f:"a b"~214748364[8]>
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:BDECA60AB7A6F547]:0)
        at app//org.apache.lucene.queryparser.flexible.core.nodes.TestSlopQueryNodeToQueryString.testMaxIntSlopRoundTrips(TestSlopQueryNodeToQueryString.java:29)

# standalone runner, seed DEADBEEF: main 645d4fe14a20 fails with -ea and -da; releases 10.6.0, 9.1.0 and 9.0.0 fail

# re-parsing the printed text (a verifier's test, same queryparser code)
printed string not re-parsable: f:"a b"~2147483648 -> java.lang.NumberFormatException: For input string: "2147483648"
  at java.lang.Integer.parseInt(Integer.java:662)
  at org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser.parseInt(StandardSyntaxParser.java:110)

# a public-API probe on main
flexible-syntax "a b"~16777217 -> node 16777217 -> printed f:"a b"~16777216 -> reparsed 16777216
flexible-StandardQueryParser "a b"~2147483647 -> PhraseQuery slop 2147483647
hand-built SlopQueryNode(16777217) -> printed f:"a b"~16777216

• The contract: QueryNode.java:26 ("convert to a query string understood by the query parser"); core/nodes/package-info.java:42-43 ("A query node tree can also generate a query string that can be parsed back by the original text parser, at this point only the standard lucene syntax is supported."). Both date from 2.9.0.
• Upstream enforces it: GITHUB#7865, fixed by #14697 (merged 2025-05-29): "AbstractRangeQueryNode#toQueryString(EscapeQuerySyntax) now returns a string which can be parsed back into the original node" (lucene/CHANGES.txt:296-297, 11.0.0 bug fixes), with TestAbstractRangeQueryNode parsing the printed string back.
• Int end to end: Integer.parseInt in the grammar, the int field, constructor and getter, StandardQueryParser.setPhraseSlop(int) and PhraseQuery.Builder.setSlop(int). Nothing documents float precision for the slop.
• Copy-paste artifact: identical to BoostQueryNode.getValueString, whose field is a float; for an int its else branch can never run.
• Tests: no upstream test prints a slop above 2^24 or asserts the current output.

The strongest case for "intended": Lucene's standard syntax has always read the slop as a float and truncated it. The classic QueryParser still does on main (QueryParserBase.java:802, (int) Float.parseFloat(...): "a b"~16777217 becomes slop 16777216), and the flexible grammar did the same until 9.0.0, so on 9.0.0 a parsed tree's printed text reads back as the same node and the float printer "matches the parser". toQueryString is also loosely honoured elsewhere (phrases in processed trees print as [TP[...]]), Lucene uses it only for messages, and the open #8667 (LUCENE-7616) calls the contract weak.

Why it fails: since 9.1.0 the flexible grammar reads an exact int, and the node is int-typed; a hand-built SlopQueryNode(q, 16777217) has never printed a value any Lucene parser reads back as 16777217; the documented contract is a re-parsable string, and upstream fixed another node for breaking it; #8667 is about custom parsers getting standard syntax back. Even on 9.0.0 the printer did not match the parser for the headline value: the parser stored 2147483647 (the (int) cast saturates float 2^31) and the printer showed ~2147483648, which read back as 2147483647 only because the cast saturated again; the minimal test fails on 9.0.0 too. The fix is also harmless under the float reading: a slop stored by a float-rounding parser prints the same digits, except the saturated Integer.MAX_VALUE, which then prints 2147483647 instead of 2147483648 and reads back as the same value. Verifiers and the adjudication rejected the intended-behaviour case; a read-only cross-model review rated it weak.

One fix was written and validated in a scratch worktree of 645d4fe14a20; the parser-side alternative is described but not run.

Replace the float detour in both getValueString() methods (+2/-6 in SlopQueryNode.java and PhraseSlopQueryNode.java). Covers toQueryString and toString of both nodes.

private CharSequence getValueString() {
  return Integer.toString(this.value);
}

Trade-offs

• Output is unchanged for every slop that printed correctly before (all of [0, 2^24] and the float-representable values above it); an exhaustive check found 0 mismatches over [0, 2^26) with the fix.
• Formatting only; not benchmarked.
• It does not change parsing: the grammar's Integer.parseInt still throws a raw NumberFormatException on a fractional or out-of-range slop, a separate parse-side issue.

What the checks validated

• Gradle, scratch worktree of 645d4fe14a20, OpenJDK 25.0.3, seed DEADBEEF (2026-10-31): stock, the minimal test fails with assertions on and off and the finder's 3-test class fails 3 of 3. With the fix: the minimal test passes with assertions on and off; the finder's class (which also covers PhraseSlopQueryNode and 16777217) passes 3 of 3; TestQueryNode passes; the whole queryparser module passes (30 classes, 533 tests, 0 failures); checkGoogleJavaFormat, ecjLintMain and forbiddenApisMain pass for the module.
• Earlier, the hunt, a verifier and the registration check validated the same change on a9779ef3c193 (7 of 7 and 3 of 3 pass), and the standalone runner on 645d4fe14a20.

Not run.

Trade-offs

Hides the printer bug for parsed trees only, leaves hand-built nodes wrong and changes accepted syntax. How the grammar should treat fractional or out-of-range slops is a separate parse-side question.

Method: semi-formal (bounded-exhaustive translation validation of the stock jar, then a public-API JUnit).

• Signal. Code reading of getValueString while writing the reference printer for a round-trip harness.
• Exhaustive check. Every int in [0, 2^31) through SlopQueryNode.toQueryString and every 64th through PhraseSlopQueryNode, against Integer.toString: 2,147,483,648 checks in 118 s on 4 threads, 2,071,986,176 mismatches, equal to the analytic count, smallest 16777217, 64 unparseable. With the fix shadowed: 0 mismatches over [0, 2^26).
• Confirmation. A 3-test public-API JUnit failed on main with assertions on and off; an independent reproducer, a devil's-advocate verifier, read-only cross-model reviews (OpenAI GPT-5.6) and an adjudication checked it, and a follow-up ran the version history, a prior-report search and a security screen.