RegExp numeric interval accepts a '+' on a bound and counts it as a digit

Lucene's RegExp has an optional numeric-interval syntax <n-m>, on by default in RegexpQuery, the query parsers' /.../ syntax, Intervals.regexp and RegexCompletionQuery. Its javadoc says the bounds are "non-negative decimal integers" and that bounds with the same number of digits fix the length of the matching strings: <1-9> matches "1" to "9", <01-09> matches "01" to "09".

A bound written with a leading + is accepted without error and the + is counted as a digit. The parser passes each bound's text to Integer.parseInt, which accepts a leading + since Java 7, and takes the width from the raw text length. So <+1-+9> is built as <01-09>: a RegexpQuery or a classic QueryParser query /<+1-+9>/ returns the term "05" instead of "5". A + on one bound only makes the lengths differ, which drops the fixed width: <+100-999> also matches "0500". Every other malformed bound tried (-1, spaces, trailing text, hex, overflow) is rejected with interval syntax error. Unsigned bounds are exhaustively correct, and every + form means something an unsigned spelling can also say, so the only effect is a silently different match set for the author's own query. It has been in every release since 4.0.0-ALPHA, observable on Java 7 or later. Requiring ASCII digits before Integer.parseInt fixes it; it passes the repro, the closest suites and :lucene:core:check -x test.

1. The bound texts. For <...> with a - (:1460-1465), a - at the start, at the end or twice is rejected (:1474), then String smin = s.substring(0, i); String smax = s.substring(i + 1); (:1476-1477).
2. Integer.parseInt accepts a +. int imin = Integer.parseInt(smin); int imax = Integer.parseInt(smax); (:1478-1479). Its javadoc allows "an ASCII plus sign '+' ... to indicate a positive value" (parseInt("+42", 10) returns 42); that is true since Java 7 (JDK-5017980).
3. The width is the text length. if (smin.length() == smax.length()) digits = smin.length(); else digits = 0; (:1480-1482), so the + is counted.
4. Built as a fixed-width interval. makeInterval(flags, imin, imax, digits) (:1488) becomes Automata.makeDecimalInterval(min, max, digits) (:728-729), where digits > 0 means a fixed number of digits and 0 means any number of leading zeros (Automata.java:490-501).
5. Every other bad bound fails. Any NumberFormatException becomes IllegalArgumentException("interval syntax error at position ...") (:1489-1490).

Over all 111,111 digit strings of length 0 to 5, <+1-+9> differs from <1-9> on 18 strings, <+1-9> on 36, and <+100-999>, <100-+999> and <+100-+999> each differ from <100-999> on 1,800. + is the only non-digit that gets through: <-1-9>, < 1-9>, <1 -9>, <1-9 >, <1-9x>, <0x1-9>, <1_0-20>, <1.0-2>, <99999999999-1>, <++1-9>, <+-1-9>, <1--9>, <+-9> and <1-+> are all rejected.

• Who is hit. A regular expression with a numeric-interval bound written with a leading +: typed by a user into a query parser, or built by an application such as "<" + lo + "-" + hi + ">" from text that carries a sign (a %+d format or a user field; Integer.toString never adds a +).
• Default reach. RegexpQuery(Term) ("By default, all regular expression features are enabled"), the classic QueryParser and the flexible StandardQueryParser (RegExp.ALL), Intervals.regexp (default flags) and RegexCompletionQuery(Term, BitsProducer) (RegExp.ALL). The classic QueryParser path was run: /<+1-+9>/ matches "05" and not "5".
• What goes wrong. No exception, and a different match set: with both bounds signed the intended values are missed and zero-padded ones match; with one bound signed any number of leading zeros is accepted.
• Limits. Deterministic, any index size, no setting. The study's bounded-exhaustive run (every <n-m> with n, m up to 1100, leading zeros and optional +, 44.4 million parses) found 0 mismatches for unsigned bounds. Every + language equals a documented spelling (<+1-+9> = <01-09>, <+100-999> = <0100-999>; 36,905 checks), and the automaton stays at most 14 states.
• Likelihood: rare. It needs a + in a bound; then it happens every time. No report since Java 7 made it reachable in 2011. RegExp and RegexpQuery are @lucene.experimental.

The test sits in the automaton test package, as Lucene's own automaton tests do, but uses no package-private member. It accepts either fix a maintainer may choose (reject the +, or read "+1" as 1): of the three readings (rejected, <1-9>, today's <01-09>) it fails only on today's. It checks "5" only; it does not assert that "05" is rejected.

As the regression test it needs two lint edits: annotate the unused catch parameter (Lucene's ecj settings make it an error, gradle/validation/ecj-lint/ecj.javadocs.prefs:136) in the layout google-java-format requires, as QueryParserBase.java:803-805 does, and shorten the 102-character comment line to at most 100:

    // <+1-+9> must be rejected or, like <1-9>, match "5"; today the '+' counts as a digit
    RegExp re;
    try {
      re = new RegExp("<+1-+9>");
    } catch (
        @SuppressWarnings("unused")
        IllegalArgumentException rejected) {
      return;
    }

With those edits it fails on stock, passes with the recommended fix, and :lucene:core:check -x test passes with it in place. The one-bound width loss (<+100-999>) and <+100-+999> share the root cause and the fix; they are left out of the test.

TestRegExpIntervalPlusSign > testPlusSignOnIntervalBounds FAILED
    java.lang.AssertionError: expected <+1-+9> to match "5" like <1-9>, but it does not (matches "05": true)
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:8022A4C09432B805]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.assertTrue(Assert.java:42)
        at org.apache.lucene.util.automaton.TestRegExpIntervalPlusSign.testPlusSignOnIntervalBounds(TestRegExpIntervalPlusSign.java:34)
:lucene:core:test (FAILURE): 1 test, 1  failure

# the same message with -Ptests.asserts=false, and on the 10.6.0 and 10.5.2 release jars

# probe on the same main, assertions off ("matches" = which of 16 probe strings are accepted):
<1-9>              ACCEPTED toString=<1-9> digits=1 matches={1,5,9}
<01-09>            ACCEPTED toString=<01-09> digits=2 matches={01,05,09}
<+1-+9>            ACCEPTED toString=<01-09> digits=2 matches={01,05,09}
<+1-9>             ACCEPTED toString=<1-9> digits=0 matches={1,5,9,01,05,09,001,005}
<100-999>          ACCEPTED toString=<100-999> digits=3 matches={100,500,999}
<+100-999>         ACCEPTED toString=<100-999> digits=0 matches={100,500,0500,00500,999,0100,0999}
<+100-+999>        ACCEPTED toString=<0100-0999> digits=4 matches={0500,0100,0999}
<-1-9>             REJECTED IllegalArgumentException: interval syntax error at position 5
<1-9x>             REJECTED IllegalArgumentException: interval syntax error at position 5
a{+2}              REJECTED IllegalArgumentException: integer expected at position 2
QueryParser /<1-9>/ -> RegexpQuery f:/<1-9>/ matches 5=true 05=false
QueryParser /<+1-+9>/ -> RegexpQuery f:/<+1-+9>/ matches 5=false 05=true

• The documented contract. "Numerical intervals are specified by non-negative decimal integers and include both end points, and if n and m have the same number of digits, then the conforming strings must have that length (i.e. prefixed by 0's)." (RegExp.java:347-350). Under neither reading of "+1" is <01-09> right.
• The block validates bounds. It rejects a misplaced - (:1474) and turns every NumberFormatException into interval syntax error (:1489-1490); the constructors document @exception IllegalArgumentException if an error occurred while parsing the regular expression (:510, :521, :533). 14 other malformed spellings are rejected.
• The sibling syntax is ASCII-digits-only. {n,m} scans peek("0123456789") (:1272-1279); a{+2} fails with integer expected at position 2.
• Tests. testTruncatedInterval (<1-), testTruncatedInterval2 (<1) and testEmptyInterval (<->) expect IllegalArgumentException (TestRegExpParsing.java:491-513); <1- and <1 fail at the > check (:1463), <-> at the bound check (:1474); no test covers a malformed bound spelling such as <1-9x>, and every interval in the tests is unsigned.
• Never designed. The parse lines came with the 2010 port of dk.brics.automaton (b679816a7001). On Java 6 Integer.parseInt("+1") threw, so <+1-+9> was an interval syntax error; JDK-5017980 (Java 7, 2011) widened parseInt afterwards (read; no Java 6 run).
• Precedent. #10410 (LUCENE-9370, type:bug): "The RegExp class is too lenient in parsing user input which can confuse or mislead users", fixed in 9.0 (its CHANGES entry is under Improvements). PR #12277 made a{99,11} throw: "we catch an invalid usage and throw a clean exception in that case".

The strongest case (reading-only cross-model reviews said not real on it): + is outside the documented <n-m> grammar of an @lucene.experimental class, so what it parses to is unspecified; every documented input is exhaustively correct; garbage in, garbage out.

Why it fails: the parser does not treat out-of-grammar bounds as unspecified; it rejects all the others tried, and its tests assert that truncated and empty intervals throw. The + gets through only because of a later JDK change, and the meaning it gets contradicts the digit-count rule even under the lenient reading. The root-cause reviewer graded its works-as-designed argument "strong", but still wrote: "The case is not conclusive because the interval documentation never formally defines the lexical production for n and m, and counting an accepted plus sign as a digit conflicts with the stated width rule." It is a good reason for low priority, and maintainers may file the fix as an improvement; it does not make <+1-+9> = <01-09> correct.

Both reject variants were run in a scratch worktree of 645d4fe14a20 (OpenJDK 25.0.3, seed DEADBEEF), removed afterwards. "Closest suites" means :lucene:core:test --tests 'org.apache.lucene.util.automaton.*' plus TestRegexpQuery, TestRegexpRandom, TestRegexpRandom2, TestAutomatonQuery and TestAutomatonQueryUnicode. The choice is the maintainers': reject the + (A or B), or read "+1" as 1 with width 1 (C, not run); the test accepts all three.

In RegExp.parseSimpleExp, after smin and smax are cut (:1476-1477), throw the existing NumberFormatException (turned into interval syntax error) unless both are non-empty runs of 0-9; a small isAsciiDigits helper, +15 lines. It matches the {n,m} syntax, which reads ASCII digits only, and makes "digits" in the width rule exact.

+  private static boolean isAsciiDigits(String s) {
+    for (int i = 0; i < s.length(); i++) {
+      char c = s.charAt(i);
+      if (c < '0' || c > '9') {
+        return false;
+      }
+    }
+    return s.isEmpty() == false;
+  }
@@ parseSimpleExp, interval branch
           String smin = s.substring(0, i);
           String smax = s.substring(i + 1);
+          // bounds are non-negative decimal integers written with the ASCII digits 0-9 only
+          // (Integer.parseInt alone would also accept a leading '+' and non-ASCII digits)
+          if (isAsciiDigits(smin) == false || isAsciiDigits(smax) == false) {
+            throw new NumberFormatException();
+          }
           int imin = Integer.parseInt(smin);

Trade-offs

• It also rejects non-ASCII decimal digit bounds, which main reads as ASCII values: <١-٩> (Arabic-Indic) and <１-９> (full-width) are accepted as <1-9> on main and rejected with this fix. That reading is defensible and not claimed as a bug, so it is a behaviour change worth a CHANGES entry; a javadoc tweak ("written with the digits 0-9") would document it.
• A query that uses a + bound today gets an exception instead of a different match set.

What the checks validated

• Stock 645d4fe14a20: the repro fails with assertions on and off.
• With the fix: the repro passes with assertions on and off; the closest suites pass (261 tests including the repro, 2 skipped); queryparser classic.* and flexible.standard.* (222 tests, 1 skipped), queries intervals.* (117) and TestRegexCompletionQuery (5) pass; :lucene:core:check -x test (google-java-format, ecj lint, forbidden APIs, license checks) passes with the lint-ready regression test in place, which passes with the fix and fails on stock.
• Probe: the five + forms are rejected with interval syntax error; the 14 other malformed spellings stay rejected; <1-9>, <01-09> and <100-999> keep their languages.

Four lines after :1477: if (smin.charAt(0) == '+' || smax.charAt(0) == '+') throw new NumberFormatException(); (both are non-empty there, :1474). The narrowest change: non-ASCII digit bounds keep today's reading.

          // Integer.parseInt accepts a leading '+', which is not a digit of a bound
          if (smin.charAt(0) == '+' || smax.charAt(0) == '+') {
            throw new NumberFormatException();
          }

Trade-offs

• The bound check still relies on Integer.parseInt's grammar, which is wider than the documented one.

What the checks validated

• The repro passes; the closest suites pass (261 tests including the repro, 2 skipped); the probe gives the same rejections as the recommended fix, and non-ASCII digit bounds stay accepted as <1-9>.

Drop one leading + from each bound before the length check (still rejecting ++1), so <+1-+9> means <1-9>. It satisfies the test, but keeps accepting a spelling no documentation mentions, while {n,m} rejects a{+2}. Not run.

Method: semi-formal (the finder's label): bounded-exhaustive translation validation of the real main jar against the javadoc, not a proof.

• The unit's plan predicted the + signal by reading the Integer.parseInt lines; the exhaustive run witnessed and measured it.
• Harness h297.IntervalTV: every <n-m> text with 0 <= n, m <= 1100, up to two leading zeros and an optional + on each bound, both orders, against a reference built from the javadoc rules; for n, m <= 120 an exact language comparison (product of the determinized Lucene automaton with the reference's minimal DFA).
• 195.3 million checks in seven legs with 0 in-domain mismatches (documented input); 35 of 35 mutants of the checker were killed, and a RegExp mutant that drops the fixed width gives 99,438 mismatches, so the check can see width errors.
• An independent reproducer then wrote its own public-API test comparing each signed form with its unsigned form over all 111,111 digit strings of length 0 to 5, and ran RegexpQuery and the classic QueryParser on an index.