DateRangePrefixTree rejects day 32 but silently rolls a day its month does not have into the next month, so [2014-11-01 TO 2014-11-31] also matches December 1st

DateRangePrefixTree (spatial-extras, @lucene.experimental) parses ISO-like date strings for NumberRangePrefixTreeStrategy. Since LUCENE-8640 (7.7.0) its parseCalendar range-checks the month, day, hour, minute and second fields (not the year or the fraction digits) and throws ParseException for day 32, month 13, hour 25 or minute 60. But the day is checked only against 1..31, never against the length of its month, and is then set on a lenient java.util.Calendar. When the fields are read, the impossible day silently becomes a real day of the next month: 2014-11-31 becomes 2014-12-01, 2014-02-30 becomes 2014-03-02, 2013-02-29 becomes 2013-03-01. The range [2014-11-01 TO 2014-11-31] parses to [2014-11 TO 2014-12-01], and an Intersects query on a three-document index returns 2014-11-15 and 2014-12-01. Both shipped template calendars give these results on these modern dates (they differ only for Julian leap years and the 1582 cutover), with assertions on and off. Every release since 5.0.0 (the first with the class) does it; the inconsistency with the day-32 check exists since 7.7.0. A 9-line month-length check makes these strings throw ParseException, keeps the deliberate hour 24 working, and passes the module's tests.

• DateRangePrefixTree: a spatial-extras prefix tree for dates and date ranges. NumberRangePrefixTreeStrategy indexes and queries its shapes; parseCalendar reads one date (2014-11-30, 2014-11-30T10, ...) and parseShape also reads a range [a TO b].
• Template calendar: the tree clones a java.util.Calendar for every value. The two shipped templates, DEFAULT_CAL and JAVA_UTIL_TIME_COMPAT_CAL, are lenient (isLenient() == true): a lenient calendar accepts November 31 and normalizes it to December 1 when it computes its fields.

parseCalendar (DateRangePrefixTree.java:469-539 at a9779ef3c193) starts from a clone of the template (:472). The month goes through parseAndCheck(str, offset, 1, 12) (:491-492), the day through parseAndCheck(str, offset, 1, 31) and then cal.set(Calendar.DAY_OF_MONTH, parsedVal) (:498-499). parseAndCheck (:548-555) only compares with the bounds it is given. Any exception inside its try block becomes a ParseException (:534-538), so day 32 is rejected; November 31 passes.

toShape(Calendar) (:308-334) reads the fields with cal.get(...) (:315-326), and toString(Calendar) (:391 on) does the same (:396-429). The first get makes the lenient calendar compute the time and rewrite its fields: November 31 becomes December 1. That is the cell that is indexed or searched.

NumberRangePrefixTree.parseShape (NumberRangePrefixTree.java:205-221) parses each bound of [a TO b] the same way (:215, through parseUnitShape, DateRangePrefixTree.java:457-459), and toRangeShape (NumberRangePrefixTree.java:117-128) normalizes [2014-11-01 TO 2014-12-01] to [2014-11 TO 2014-12-01]: all of November plus December 1st.

Each accepted day past its month's length becomes a real date at most 3 days after the named month's end (2014-02-31 -> 2014-03-03); the 10 cutover-gap days of DEFAULT_CAL move 10 days forward inside October (1582-10-10 -> 1582-10-20). DEFAULT_CAL uses Julian leap years before 1582, so 1500-02-29 is real there and rolls only under the proleptic JAVA_UTIL_TIME_COMPAT_CAL. Every accepted string parses to exactly the shape of the real date it rolls to (101 of 101 and 93 of 93); for three sample pairs per template calendar (a day, a millisecond timestamp, a range) it also indexes the same terms as that date, with CheckIndex clean (the hunt's security probe, main and 10.5.2).

• Who. Applications that use DateRangePrefixTree with NumberRangePrefixTreeStrategy and parse date strings that people type or code builds. A range written as [2014-02-01 TO 2014-02-31] covers 2014-03-01..03; code that builds month ends by appending -31 covers the next month's first day for every 30-day month. A document value 2014-11-31 is indexed as 2014-12-01.
• Downstream (code reading only, not executed): Solr's DateRangeField builds this tree on JAVA_UTIL_TIME_COMPAT_CAL and sends range strings to parseShape, and single values without Z that do not start with NOW to parseCalendar, for documents and queries. Values with Z go through Solr's own date parser, which rejects these dates. Other downstream search engines checked do not use this class.
• Opt-in, experimental. The spatial-extras module is opt-in; DateRangePrefixTree and NumberRangePrefixTreeStrategy are @lucene.experimental (DateRangePrefixTree.java:39, NumberRangePrefixTreeStrategy.java:42). Within it nothing special is needed: either shipped template calendar, any IndexWriterConfig and codec, one value.
• Silent and small. No exception and no index damage; the value means a different real date, at most 3 days past the named month's end. Deterministic; the same with assertions off.
• How likely. Rare: standard date pickers and formatters normally do not produce November 31 (stored or hand-built strings can). Registry likelihood "rare", priority low.
• Workaround today. Validate the date part first, for example with java.time.LocalDate.parse for modern dates (it rejects these strings). A non-lenient template calendar is no clean way out: 2014-11-31 then fails with IllegalArgumentException: MONTH: 10 -> 11 from toString/parseShape, later than and unlike the fixed-field checks, which throw ParseException at parse time; and the deliberate T24 is rejected too.

# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
    => java.lang.AssertionError: expected ParseException for 2014-11-31 (as for 2014-11-32), got 2014-12-01
       org.junit.Assert.fail(Assert.java:89)
       org.apache.lucene.spatial.prefix.tree.TestDateRangeNonexistentDay.testNovember31IsRejected(TestDateRangeNonexistentDay.java:33)

• The method rejects every other out-of-range field value. LUCENE-8640 ("Date Range format validation", CHANGES.txt:6973, 7.7.0 section) added parseAndCheck for the month, day, hour, minute and second (DateRangePrefixTree.java:491, 498, 505, 512, 519; the year and the fraction digits are not range-checked). Day 32, month 13, hour 25 and minute 60 throw ParseException (executed controls). The upstream TestDateRangePrefixTree.testInvalidDateException (TestDateRangePrefixTree.java:282-353) asserts it for month 13 and hour 25. November 31 shows the same symptom as November 32, a silent move into December, yet only one is rejected.
• The formatter it reverses cannot emit it, and the analogous ISO parser rejects it. No javadoc says how impossible input is handled, but parseCalendar is "The reverse of toString(java.util.Calendar)" (DateRangePrefixTree.java:462); toString(Calendar) is "consistent with DateTimeFormatter#ISO_INSTANT" (:386). Instant.parse rejects 2014-11-31T00:00:00Z and 2014-02-29T00:00:00Z and accepts 2014-11-30T24:00:00Z as the next midnight, the rollover the parser allows on purpose (hours 0..24). The parser's looser T24:30 (read as 00:30 of the next day) is rejected by Instant.parse and is not part of this claim.
• The range parser promises an exception: parseShape "@throws ParseException If there is a problem" (NumberRangePrefixTree.java:203).
• Nothing asks for the rollover. No upstream test parses a day missing from its month. LUCENE-8640's issue and PR review discuss delimiters, cost and hour 24 ("We might have hours from 0 to 24 inclusively"), never month length or leap years.
• The LUCENE-8640 comment "reject it absolutely since we parting a range not a date" was about malformed delimiters, not month length.

The strongest case for "works as designed": no javadoc sentence says impossible dates are rejected; parseCalendar is specified only as the reverse of toString, whose output never contains 2014-11-31, so this is out-of-contract input and a fix is hardening. The class is Calendar-based and @lucene.experimental, a Calendar is lenient by default, the template is configurable (LUCENE-7278), LUCENE-8640 was an enhancement about syntax, and other Lucene date parsers are lenient too (DateTools.stringToDate reads 20140230 as 2014-03-02; the classic and flexible query parsers call setLenient(true)).

Why it fails: the parser itself already rejects every other out-of-range field value (day 32, month 13, hour 25, minute 60), whatever the lenient calendar would do with it; the rollover it keeps on purpose, hour 24, is one ISO also accepts for T24:00; and the "use a strict calendar" way out is undocumented, surfaces the error later as an IllegalArgumentException instead of the ParseException the fixed-field checks give at parse time, and also rejects hour 24. @lucene.experimental lowers the severity, not the inconsistency. The adjudication confirmed with medium confidence for this reason: the mechanism is certain, but maintainers may reasonably call the fix validation hardening.

One fix was applied in a scratch worktree at a9779ef3c193 and run through gradle (seed DEADBEEF) and through the classpath sweep; the alternatives were not run.

After parsing the day, take a clone of the calendar, set the day to 1 and compare the parsed day with getActualMaximum(DAY_OF_MONTH); throw IllegalArgumentException, which the existing catch turns into ParseException like the other field checks. The clone keeps the isSet() precision state that toShape relies on (the class javadoc warns that any get sets every field, DateRangePrefixTree.java:35-37). The finder wrote this fix; the independent reproducer and the adjudication ran it too.

--- a/lucene/spatial-extras/src/java/org/apache/lucene/spatial/prefix/tree/DateRangePrefixTree.java
+++ b/lucene/spatial-extras/src/java/org/apache/lucene/spatial/prefix/tree/DateRangePrefixTree.java
@@ -496,6 +496,15 @@ public class DateRangePrefixTree extends NumberRangePrefixTree {
       checkDelimiter(str, offset - 1, '-');
 
       parsedVal = parseAndCheck(str, offset, 1, 31);
+      // reject a day that does not exist in this month (e.g. Feb 30); a lenient Calendar would
+      // otherwise roll it over into the next month. Checked on a clone to keep cal's isSet() state.
+      Calendar monthCal = (Calendar) cal.clone();
+      monthCal.set(Calendar.DAY_OF_MONTH, 1);
+      int daysInMonth = monthCal.getActualMaximum(Calendar.DAY_OF_MONTH);
+      if (parsedVal > daysInMonth) {
+        throw new IllegalArgumentException(
+            "Invalid value: " + parsedVal + ", expecting from 1 to " + daysInMonth + "]");
+      }
       cal.set(Calendar.DAY_OF_MONTH, parsedVal);
       offset += 3;
       if (lastOffset < offset) return cal;

Trade-offs

• Invalid input only: strings that name a non-existent day now throw ParseException, for queries and for documents. An application that relied on the rollover (for example appending -31 for "end of month") gets an error instead of a slightly wrong range; worth a CHANGES entry. Every real date in the sweep parses as before: 5,107 (DEFAULT_CAL) and 5,115 (JAVA_UTIL_TIME_COMPAT_CAL) of 5,376 day-precision strings per template round-trip before and after. Other precisions and years were not swept (the module's 120 tests pass with the fix).
• Calendar-aware: getActualMaximum follows the template's calendar system, so 1500-02-29 stays valid under DEFAULT_CAL (Julian leap year) and is rejected under the proleptic JAVA_UTIL_TIME_COMPAT_CAL. A java.time-based check would get the first case wrong.
• Shipped templates only: validated with DEFAULT_CAL and JAVA_UTIL_TIME_COMPAT_CAL; the constructor accepts any caller-supplied Calendar template, and custom Calendar subclasses were not tested.
• Leaves the 1582 cutover gap: under DEFAULT_CAL, 1582-10-05..1582-10-14 (days the hybrid calendar skips) still move forward 10 days. Out of scope here; the round-trip option covers it.
• Keeps hour 24 (2014-11-30T24 -> 2014-12-01T00) and the looser T24:30 form unchanged.
• Cost: one Calendar clone and one getActualMaximum per parsed value with a day; parse path only. Not benchmarked.
• No reindexing: indexes built from valid strings are unchanged; a value indexed earlier from an impossible string keeps the date it rolled to.
• The message mirrors parseAndCheck's, including its stray ]; a PR may tidy both.

What the checks validated

• Stock (scratch worktree, gradle, seed DEADBEEF): the repro fails 2 of 3 with assertions on and off (testControls passes); the 3 closest suites (TestDateRangePrefixTree, TestDateNRStrategy, TestNumberRangeFacets) pass, 20 tests.
• With the fix: the repro passes 3 of 3 with assertions on and off; the 3 closest suites pass, 20 tests; the whole spatial-extras test suite with the repro in place passes, 27 classes, 120 tests, 5 skipped, 0 failures; TestDateRangePrefixTree and TestDateNRStrategy with 20 iterations pass, 380 tests; ecjLintMain, ecjLintTest and checkGoogleJavaFormat of the module pass.
• Sweep (classpath runner, main jars with the fixed class shadowed): every non-existent day with JAVA_UTIL_TIME_COMPAT_CAL (93 of 93) and every month-length case with DEFAULT_CAL (91 of 91) now throws ParseException; all 5,107 / 5,115 real dates still round-trip; 2014-11-30T24 still reads 2014-12-01T00; 1500-02-29 stays valid under DEFAULT_CAL only. The 10 cutover-gap days of DEFAULT_CAL still roll.
• Earlier hunt runs with the same fix: the finder's test passes and the upstream date suites pass 100 of 100; the independent reproducer's 3 tests pass 3 of 3 (also in the adjudication's re-run).

After parsing, compare every parsed field with what a clone's get() returns and throw on any difference, special-casing hour 24.

Trade-offs

Catches every rollover, including the 10 cutover-gap days of DEFAULT_CAL, but is more code (the method returns early after each field) and must keep hour 24 and the isSet() state right. Not run.

Call setLenient(false) on the working calendar.

Trade-offs

The error then surfaces later, as an IllegalArgumentException (MONTH: 10 -> 11) from toString/toShape instead of a ParseException at parse time, and the deliberate hour 24 is rejected too (executed by the verifiers on stock main with a strict template). Not recommended.

State in the parseCalendar and parseShape javadoc that a day past the month's end rolls into the next month.

Trade-offs

No behaviour change; leaves the inconsistency with day 32 and the silent extra matches.

Check the date part with java.time.LocalDate.parse before passing the string (it rejects 2014-11-31 and 2014-02-29).

Trade-offs

java.time is proleptic Gregorian, so for dates before 1582 under DEFAULT_CAL it disagrees with the tree's own calendar.

Method: code reading (non-formal tier), in a hunt unit on spatial-extras range trees and counters.

• The unit was building a bounded-exhaustive harness for DateRangePrefixTree with a java.time reference model, which rejects dates such as 2014-02-30. Reading parseCalendar while writing it showed that the day is checked only against 1..31.
• A direct public-API probe showed the parser accepts it and rolls it into March; a JUnit on the stock main jars, with the day-32 rejection as its control, then failed with assertions on and off, and on 10.5.2 and 9.12.3.
• It was then checked by an independent reproducer, an intended-behaviour devil's advocate, read-only reviews by a model from another family (OpenAI GPT-5.6), an adjudication, a history, prior-report and security follow-up, and an audit. Found 2026-10-26 on main a9779ef3c193.