Points-only date range strategy: Intersects on a year, month or day matches nothing

NumberRangePrefixTreeStrategy over DateRangePrefixTree (or a custom NumberRangePrefixTree) with the opt-in setPointsOnly(true) returns no documents for an Intersects query whose shape is one unit coarser than full precision: a year, month, day, hour, minute or second, or a range such as [2014-10-01 TO 2014-10-31] that the tree normalizes to one unit. One document at 2014-10-23T21:22:33.159 is found by IsWithin with 2014-10 (1 hit) and by Intersects with pointsOnly=false (1 hit), but Intersects with pointsOnly=true gives 0. RecursivePrefixTreeStrategy.makeGridShapeIntersectsQuery builds a single TermQuery on the unit's term plus the leaf marker byte, and points-only indexing writes leaf terms only at full precision; every coarser level of a point is stored as a prefix (non-leaf) term. The 7.0.0 CHANGES entry of LUCENE-7845 (d4f87b4a36c) introduced this shortcut as an optimization for "a simple date interval (e.g. 1 month)", exactly the failing input. No error is raised; the index is unaffected. Every release from 7.0.0 to 10.5.2 and main fails the same way, and 10.6.0 has the same code (read); a one-statement fix that queries the non-leaf term for a coarse cell passes the repro and the module's whole test suite.

• Number/date range prefix tree: DateRangePrefixTree splits time into levels: 1 million years, 1,000 years, year (level 3), month (4), day (5), hour (6), minute (7), second (8) and millisecond (9, the maximum). A shape such as 2014-10 is a unit: one cell of the tree.
• Indexing: a shape is indexed as one term per cell. A cell that the shape covers completely is a leaf and its term gets a trailing leaf byte (NumberRangePrefixTree.java:690-698); a cell the shape only partly covers is a prefix term without it. A full-precision instant therefore gives 8 prefix terms (levels 1-8) and 1 leaf term (level 9).
• setPointsOnly(true): tells the strategy that only points (full-precision instants) are indexed; any other shape is rejected at indexing time (PrefixTreeStrategy.java:176-182). Its javadoc: "There are no \"leaves\" in such a case, except those at maximum precision" (:119-125).

1. NumberRangePrefixTreeStrategy.isGridAlignedShape is true for any unit with level > 0 (NumberRangePrefixTreeStrategy.java:68-76), so RecursivePrefixTreeStrategy.makeQuery sends Intersects on a unit to makeGridShapeIntersectsQuery (RecursivePrefixTreeStrategy.java:185-188).
2. Under pointsOnly that method walks the query shape's cells to the last one, the unit's own cell, which the iterator marks as a leaf because the unit covers it (NumberRangePrefixTree.java:858-864), and returns TermQuery(cell.getTokenBytesWithLeaf(null)): the unit's term plus the leaf byte (RecursivePrefixTreeStrategy.java:218-229).
3. Points-only indexing at full precision never writes a leaf term above full precision (this tree always indexes at the maximum level, NumberRangePrefixTree.java:479-487), so the query matches nothing. The term that is there, the same bytes without the leaf byte, is held by every point inside the unit. Only the public overload createIndexableFields(Shape, int detailLevel) with a coarser level can write a coarse leaf, against the setPointsOnly javadoc; such a document is the only kind the shortcut finds (executed).
4. A range that covers exactly one unit is normalized to that unit (NumberRangePrefixTree.java:107-117) and fails too. Spans such as [2014 TO 2015] use IntersectsPrefixTreeQuery and are correct. In the 2D trees the query cell is always at the maximum level, so they are not affected.

One document at 2014-10-23T21:22:33.159: IsWithin = 1 and Intersects = 0 for 2014, 2014-10, [2014-10-01 TO 2014-10-31], 2014-10-23 and 2014-10-23T21:22:33; with pointsOnly=false all are 1. Five documents (an independent test): 2014 expected 3 got 0, 2014-10 expected 2 got 0, the day to second units expected 1 got 0. The queried term 2014-10 + leaf byte has docFreq 0; 2014-10 without it has docFreq 1.

• Who is exposed. Applications that use NumberRangePrefixTreeStrategy directly and call the non-default setPointsOnly(true) (the constructor sets false, NumberRangePrefixTreeStrategy.java:50). For them every Intersects query on one coarse unit, or on a range that normalizes to one unit, returns 0 hits. That is the optimization's headline query, so a user of the setting meets it on the first month or day query (a judgement, not measured).
• Not affected. pointsOnly=false, full-precision points, spans, IsWithin and Contains (other query classes), and the 2D strategies.
• Silent, any scale. No exception or warning; one document reproduces it; assertions on or off.
• The index is fine. For the same full-precision dates the flag does not change the indexed terms (executed), so no reindex is needed after a fix. Workaround until then: IsWithin, or a second strategy instance with pointsOnly=false at query time; both work on the existing index (executed). IsWithin means the same only while the field really holds points and runs WithinPrefixTreeQuery with the setting the code calls "slower but ensures correct results" (RecursivePrefixTreeStrategy.java:191-198); the second strategy runs the general IntersectsPrefixTreeQuery. Neither was benchmarked.
• Downstream (code reading only, not executed). In their default branches as read on 2026-10-26, Solr's DateRangeField builds this strategy but never calls setPointsOnly, and Elasticsearch and OpenSearch do not use it; older downstream releases and third-party plugins were not checked. So none of these servers exposes the setting today.

# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
    => java.lang.AssertionError: expected:<1> but was:<0>
       org.junit.Assert.fail(Assert.java:89)
       org.junit.Assert.failNotEquals(Assert.java:835)
       org.junit.Assert.assertEquals(Assert.java:647)
       org.junit.Assert.assertEquals(Assert.java:633)
       org.apache.lucene.spatial.prefix.TestNumberRangePointsOnlyUnitIntersects.testPointsOnlyIntersectsMonth(TestNumberRangePointsOnlyUnitIntersects.java:49)

• The CHANGES entry names the failing input. CHANGES.txt:8140-8142 (7.0.0): "LUCENE-7845: Enhance spatial-extras RecursivePrefixTreeStrategy queries when the query is a point (for 2D) or a is a simple date interval (e.g. 1 month). When the strategy is marked as pointsOnly, the results is a TermQuery." The issue (#8896): "if the query is a grid cell (a common date range unit like some particular day) ... If the data is pointsOnly the strategy can return a TermQuery". The code comment: "this will be equivalent to a TermQuery" (RecursivePrefixTreeStrategy.java:219).
• The setter states the invariant the shortcut breaks. PrefixTreeStrategy.java:119-125: "True if only indexed points shall be supported. There are no \"leaves\" in such a case, except those at maximum precision."
• The class javadoc promises the equivalence that fails. PrefixTreeStrategy.java:51-53: "Can query with any shape." and "If only points are indexed then this is effectively equivalent to IsWithin."
• The combination is meant to work. The introducing commit d4f87b4a36c made setPointsOnly(true) usable with this strategy and made TestDateNRStrategy randomize it (TestDateNRStrategy.java:41). That test's points-only documents are random instants over the whole long range (:94-102) while queries stay near 1970 (:43-45, :129-131), so a point almost never falls inside a coarse query unit and 0 hits is also the expected answer there.
• Precedent. Two other wrong results from the same optimization were fixed as bugs: LUCENE-7859 (#8910) and 6b022c98f9a.

The strongest case for "intended": a leaf term on a coarse cell could mean "a document indexed exactly as that unit", so the TermQuery would be a precise exact-unit match; the strategy is @lucene.experimental and pointsOnly is opt-in; the class javadoc still says "Only Intersects is supported"; and it calls the strategy "most effective as a fast approximate spatial search filter" (PrefixTreeStrategy.java:37-40).

Why it fails: under pointsOnly, indexing 2014-10 throws, and a point indexed the normal way never gets a coarse leaf term (the tree always indexes at full precision). The only documents the exact-unit reading could match are points indexed through the createIndexableFields(Shape, int detailLevel) overload at a coarser level, against the setPointsOnly javadoc; for every normally indexed point the query never matches, against the CHANGES entry that calls it an optimization for one month. The "approximate" wording is about the grid precision of shape boundaries; this strategy sets setDistErrPct(0) and its tree says "We don't do approximations in this tree/strategy", and losing every hit of a unit is not an approximation. @lucene.experimental is about API stability, not about returning nothing. "Only Intersects is supported" is older wording (the strategy also supports IsWithin and Contains), and the same bullet promises that points-only Intersects equals IsWithin, which is what fails. The read-only cross-model reviewers and the devil's advocate rated the intended case weak. Verdict: a defect.

One fix was validated through gradle and the standalone runner; an alternative was validated through the standalone runner only; documenting the behaviour is not recommended.

In RecursivePrefixTreeStrategy.makeGridShapeIntersectsQuery, under pointsOnly, use cell.getTokenBytesNoLeaf when cell.getLevel() < grid.getMaxLevels() and keep getTokenBytesWithLeaf at the maximum level. 1 file, +9/-1. Exact for points indexed the normal way (at full precision): each such point stores the non-leaf term of each ancestor cell, so the unit's non-leaf term is held by exactly those points inside it.

assert cell.isLeaf();
// pointsOnly: only cells at the maximum level are indexed as leaves; a coarser grid-aligned
// shape (e.g. a date unit coarser than full precision) is indexed as a prefix (non-leaf) term
// of every point beneath it.
return new TermQuery(
    new Term(
        getFieldName(),
        cell.getLevel() == grid.getMaxLevels()
            ? cell.getTokenBytesWithLeaf(null)
            : cell.getTokenBytesNoLeaf(null)));

Trade-offs

Keeps the single TermQuery LUCENE-7845 wanted and needs no prefix-tree traversal. Today's query looks up a term with no postings; the fixed one reads the unit's postings, one per point inside the unit, so its cost grows with the matches (no benchmark was run). The maximum-level case and the 2D strategies (always at the maximum level) are unchanged. It relies on the pointsOnly contract: on an index that broke it, results could be incomplete, as with any pointsOnly shortcut. Two concrete cases: an index written by a pointsOnly=false strategy with ranges, and points indexed through the createIndexableFields(Shape, int detailLevel) overload at a coarser level. The latter are found today, but fix A no longer finds them; option B finds both kinds (fact check's probe, one point indexed each way: stock 1, fix A 1, option B 2, IsWithin 2 of 2). It changes the base class RecursivePrefixTreeStrategy, which external code can subclass; in Lucene only NumberRangePrefixTreeStrategy widens isGridAlignedShape.

What the checks validated

In a scratch worktree through gradle: checkGoogleJavaFormat passes; the module's whole test suite (all 27 test classes, including the new test) runs 119 tests with 0 failures and 5 skips (BBox and Geo3d suites); the new test passes with assertions on and off. Through the standalone runner: the new test passes with assertions on and off. Earlier, the same change made the finder's, the independent reproducer's and the adjudication's tests, 100 upstream test runs and the security probe pass, and removed every pointsOnly mismatch of the exhaustive harnesses (with-fix runs with assertions on). An independent fact check re-ran the module's suite with the fix through gradle: 119 tests, 0 failures.

In NumberRangePrefixTreeStrategy.isGridAlignedShape, return isPointShape(shape) when isPointsOnly(), so a coarse unit takes IntersectsPrefixTreeQuery, as spans already do.

Trade-offs

Correct for both kinds of indexed points (normal and coarse-level overload) and confined to the number/date strategy, leaving the base class alone, but it drops the optimization for the "1 month" case LUCENE-7845 was written for: the general query walks the unit's cells and scans terms, which costs more than one term lookup, more so for a year. Not benchmarked.

What the checks validated

The finder's copy of the repro (the same test before formatting), the independent reproducer's test and the security screen's probe pass with it (assertions on); in the fact check's overload probe it counts 2 of 2. Not run through gradle or the module's suite.

Declare coarse units unsupported under pointsOnly.

Trade-offs

Contradicts the CHANGES entry, makes the shortcut dead code for coarse units and leaves silent 0-hit answers in place.

Method: bounded-exhaustive translation validation of the real jar against independent references (the hunt's semi-formal tier).

• Six small NumberRangePrefixTree test subclasses, including variable fan-out, were compared with a 1-D interval reference over every unit, every [a TO b] range and every operation, prefix grid scan level and pointsOnly setting: 292,244,615 checks. In pointsOnly mode the end-to-end stage reported 1,880 Intersects false negatives, all on a unit query coarser than the maximum level.
• DateRangePrefixTree was compared with a java.time reference: 10,679,369 checks, 410 pointsOnly Intersects false negatives.
• Both groups drop to 0 with the fix. The stock runs used assertions on and off; the with-fix runs used assertions on only, and the date harness's with-fix run also applied a second fix for a separately tracked mismatch class. The root cause was then read from the code and reduced to a one-document public-API JUnit. (The same harnesses also found other, separately tracked mismatches.)