DoubleRange/LongRange#getQuery never rewrites its values source, so DoubleValuesSource.fromQuery throws "must be rewritten"

DoubleRange#getQuery(Query, DoubleValuesSource) and LongRange#getQuery(Query, LongValuesSource) return a private ValueSourceQuery whose rewrite rewrites only the fast-match query and whose createWeight calls valueSource.getValues(context, null) and valueSource.isCacheable(ctx) on the source exactly as given (DoubleRange.java:156-164, :190, :210; LongRange.java:143-150, :177, :197 on main 645d4fe14a20). Nothing in either class calls valueSource.rewrite(searcher), although DoubleValuesSource says "Queries that use DoubleValuesSource objects should call rewrite() during Query#createWeight" (DoubleValuesSource.java:97-99) and FunctionMatchQuery does (FunctionMatchQuery.java:88). So DoubleValuesSource.fromQuery (and its toLongValuesSource() view) and IndexReaderFunctions sources throw "must be rewritten" on every search that reaches a segment's values, and a legacy QueryValueSource view silently scores with its own default-Similarity searcher. Introduced in 7.2.0 by LUCENE-7736, which added the rewrite rule and updated FunctionMatchQuery and FunctionScoreQuery but not these queries; it was executed on 10 releases from 7.2.0 to 10.6.0 and on main, the other release tags read the same, and 8.8.2-8.11.3 (not tagged in the history clone) are inferred. Field and expression-of-fields sources, the documented use, are unaffected, and source.rewrite(searcher) works around it. Intent is contested: the facet module documents that its Multi* wrappers never call rewrite, and the wrapped overload builds this same query, so the doc offers a javadoc fix as well as a code fix. A fix that rewrites once in createWeight passes the repro and the facet module's checks; it must land with the fix for LFV-770, or sortable-long sources regress.

1. The query never rewrites its source. getQuery(Query, DoubleValuesSource) returns new ValueSourceQuery(this, fastMatchQuery, valueSource) (DoubleRange.java:335-337). Its rewrite rewrites only fastMatchQuery (:156-164). Its createWeight (:166-213):

   final DoubleValues values = valueSource.getValues(context, null);   // :190, the source as given
   ...
   public boolean isCacheable(LeafReaderContext ctx) {
     return valueSource.isCacheable(ctx);                               // :210
   }

LongRange.java is the same (getQuery :322, rewrite :143-150, getValues :177, isCacheable :197).

1. Core's rule. The class javadoc: "clients should call rewrite(IndexSearcher) against the top-level searcher, and then call getValues on the resulting DoubleValuesSource" (DoubleValuesSource.java:34-36; LongValuesSource.java:30-32). And on rewrite: "Queries that use DoubleValuesSource objects should call rewrite() during Query#createWeight(IndexSearcher, ScoreMode, float) rather than during Query#rewrite(IndexSearcher) to avoid IndexReader reference leakage" (:97-99). FunctionMatchQuery.java:88 does DoubleValuesSource vs = source.rewrite(searcher); in createWeight; so do FunctionScoreQuery.java:121 and the sandbox CoveringQuery.java:172 (for its LongValuesSource), while RescoreTopNQuery.java:68-69 rewrites in Query#rewrite, the place the rule says to avoid. The one other Query that holds a values source and skips the step, spatial-extras' private PointVectorStrategy.DistanceRangeQuery (PointVectorStrategy.java:277), only ever gets the strategy's own DistanceValueSource, whose rewrite returns this. The Query sentence is on DoubleValuesSource#rewrite only: LongValuesSource#rewrite (LongValuesSource.java:64-70) lacks it, so the LongRange half rests on the class protocol and the parallel design.
2. Sources that throw. fromQuery returns a source whose getValues is throw new UnsupportedOperationException("This DoubleValuesSource must be rewritten") (DoubleValuesSource.java:741, :768); its rewrite builds a searcher-bound source (:777-779). Its toLongValuesSource() view delegates to it. The index-statistic sources of IndexReaderFunctions, such as maxDoc() and sumTotalTermFreq, throw "IndexReaderFunction must be rewritten before use" (IndexReaderFunctions.java:320, :99); its per-document termFreq does not.
3. The silent legacy path. ValueSource#asDoubleValuesSource() returns new WrappedDoubleValuesSource(this, null) (ValueSource.java:184-186), so its values context holds a null searcher until rewrite(searcher) makes a copy that holds one (:199-200, :244-246). QueryValueSource with a null searcher builds new IndexSearcher(...) over the top-level reader (QueryValueSource.java:114-125), with the default Similarity.

• range.getQuery(null, DoubleValuesSource.fromQuery(f:a)), range (0, +inf]: throws "must be rewritten" at DoubleRange.java:190; FunctionMatchQuery and getQuery(null, score.rewrite(searcher)) count 1. The document's BM25 score is 0.1308.
• LongRange.getQuery(null, score.toLongValuesSource()): throws at LongRange.java:177. getQuery(null, IndexReaderFunctions.maxDoc()): "IndexReaderFunction must be rewritten before use".
• ClassicSimilarity searcher, new QueryValueSource(f:a, 0f).asDoubleValuesSource(), range [1.0, 1.0] (the top Classic score): the unrewritten view computes 0.1308 (BM25), so the range query counts 0; FunctionMatchQuery and the rewritten view count 1. (The hunt's 5-document test: 0 instead of 3.)
• Control: DoubleValuesSource.fromLongField("n") counts 1; its rewrite returns itself.

• Who can hit it. Applications that build a range filter or drill-down with DoubleRange#getQuery(Query, DoubleValuesSource) or LongRange#getQuery(Query, LongValuesSource) (both classes @lucene.experimental) over a source that needs rewriting: DoubleValuesSource.fromQuery or its toLongValuesSource() view, an IndexReaderFunctions statistic, or a searcher-dependent legacy ValueSource exposed with asDoubleValuesSource().
• Not hit. Field sources and expressions over fields: the facet demo drills down with range.getQuery(boundingBox, vs) over a haversine expression of two fields (DistanceFacetsExample.java:239-243).
• What it takes. Public API, default codec, IndexWriterConfig and searcher, one document. The silent variant also needs a non-default Similarity on the searcher.
• Effect. Mostly a deterministic UnsupportedOperationException, whose message names the missing step, on every search that reaches the values (DoubleRange.java:190): any segment, or with a fast-match query any segment where that query matches a document. The silent variant gives a wrong filter result. No corruption or hang was seen in the cases run.
• How likely. Rare. An application that also counts such ranges must already pre-rewrite the source for the range counters, which throw on it too, and would often reuse that source in the drill-down. The recorded searches found no public report since 7.2.0 (2017); their gaps are under Prior reports.
• For searcher-bound sources the workaround binds the query to one searcher. getQuery(fastMatch, source.rewrite(searcher)) works. The result is "specialised for the given IndexSearcher" (DoubleValuesSource.java:92); for a source whose rewrite captures the searcher, such as fromQuery (its rewrite builds a Weight, :777-779), the query has to be rebuilt after a reopen; core's rule exists "to avoid IndexReader reference leakage". Field sources, whose rewrite returns this, have no such cost.
• Out of scope: score-dependent sources. The query passes null scores to getValues today and with the fix (DoubleRange.java:190), so DoubleValuesSource.SCORES and similar sources ("callers must not pass null", DoubleValuesSource.java:350-354) are unsupported either way; neither the bug nor the fixes are about them.

The test catches the UnsupportedOperationException only to turn it into an assertion that states expected versus actual, with the exception as the cause. The body has one document, no threads and no random calls (ByteBuffersDirectory, default IndexWriterConfig, new IndexSearcher(reader)). The seed still drives the test framework's per-run settings; the result was the same in every recorded run (seeds DEADBEEF and 5EED1234, gradle and the standalone runner). The file carries the Apache license header; with the fix applied, :lucene:facet:check passes with it included unchanged, so it can go into a PR as is.

Left out on purpose: the LongRange view, IndexReaderFunctions, the silent QueryValueSource variant and the FunctionMatchQuery comparison; they are in the explanation above.

cp $REPRO/TestDoubleRangeGetQueryRewritesSource.java $LUCENE/lucene/facet/src/test/org/apache/lucene/facet/range/
cd $LUCENE && ./gradlew :lucene:facet:test --tests TestDoubleRangeGetQueryRewritesSource -Ptests.seed=DEADBEEF -Ptests.asserts=true
# also with -Ptests.asserts=false

TestDoubleRangeGetQueryRewritesSource > testGetQueryOverQueryScoreSource FAILED
    java.lang.AssertionError: expected count 1, got java.lang.UnsupportedOperationException: This DoubleValuesSource must be rewritten
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:31099228EEA9893F]:0)
        at org.apache.lucene.facet.range.TestDoubleRangeGetQueryRewritesSource.testGetQueryOverQueryScoreSource(TestDoubleRangeGetQueryRewritesSource.java:48)

        Caused by:
        java.lang.UnsupportedOperationException: This DoubleValuesSource must be rewritten
            at org.apache.lucene.search.DoubleValuesSource$QueryDoubleValuesSource.getValues(DoubleValuesSource.java:768)
            at org.apache.lucene.facet.range.DoubleRange$ValueSourceQuery$1.scorerSupplier(DoubleRange.java:190)
            ...
            at org.apache.lucene.search.IndexSearcher.count(IndexSearcher.java:528)
:lucene:facet:test (FAILURE): 1 test, 1  failure

# the same with -Ptests.asserts=false; the same on the 10.6.0 release jars (assertions on and off)
# with the fix: :lucene:facet:test (SUCCESS): 1 test (assertions on and off)
# abridged: gradle prints the core frames with an org.apache.lucene.core@11.0.0-SNAPSHOT/ module prefix, and some frames are left out (...)

This is a conflict between two written contracts: core's rule for Queries and the facet module's documented no-rewrite design for its Multi* wrappers, which the wrapped getQuery overload shares at run time. The adjudication rated it a defect (medium confidence), the devil's-advocate verifier called intent "uncertain", and a cross-model fact check of this doc rated the case for "intended" strong, though not decisive.

• Contract. DoubleValuesSource.java:97-99 (above); the client protocol at :34-36 and LongValuesSource.java:30-32. ValueSourceQuery is an ordinary Query that receives the IndexSearcher in createWeight. The Query sentence is on DoubleValuesSource#rewrite only; the LongRange half rests on the LongValuesSource class protocol and the parallel design (the sandbox CoveringQuery rewrites its LongValuesSource in createWeight, CoveringQuery.java:172).
• The getQuery javadoc (DoubleRange.java:323-334, LongRange.java:310-321): "Create a Query that matches documents in this range ... If the value source is static, eg an indexed numeric field, it may be faster to use PointRangeQuery". No restriction on the source; non-static sources are implied.
• History. The getValues(context, null) line is from LUCENE-7610 (master 713b65d1dcc, branch_6x a238610bab1, both committed 2017-01-07; first release 6.4.0, from the branch_6x commit), before any rewrite step existed. LUCENE-7736 (#8787; 03afeb7766a on branch_7x, 2a4dd499bbd on master, committed 2017-11-15) added rewrite() and the rule, added the call to FunctionMatchQuery and FunctionScoreQuery, and in the facet module changed only a test source (rewrite() { return this; }). In its review Adrien Grand wrote "I think you forgot to update FunctionMatchQuery to make it rewrite the values source?", and Alan Woodward answered "I did, thanks for picking it up"; CoveringQuery was fixed in a follow-up commit (3f0b240853d). The facet range queries are not mentioned.
• Other Queries. Every other class in Lucene that extends Query and takes a caller-supplied DoubleValuesSource or LongValuesSource rewrites it: FunctionMatchQuery.java:88, FunctionScoreQuery.java:121 and CoveringQuery.java:172 in createWeight, RescoreTopNQuery.java:68-69 in Query#rewrite. The only other one that skips the step, spatial-extras' private PointVectorStrategy.DistanceRangeQuery, only ever gets its own DistanceValueSource, whose rewrite returns this (PointVectorStrategy.java:197-198, :277; DistanceValueSource.java:96-99).

The strongest case for "intended" (built by the devil's-advocate verifier and strengthened by the cross-model fact check): the facet module documents a no-rewrite design. MultiDoubleValuesSource.java:41-47 says "DoubleValuesSource#rewrite(IndexSearcher) will also never be called. The user should be aware of this if using a DoubleValuesSource that relies on rewriting or scores. The faceting use-cases don't call rewrite or provide scores, which is why this simplification was made." (MultiLongValuesSource.java:43-49 likewise.) In PR #543 (LUCENE-10245) Alan Woodward wrote "I don't think we need needsScores() or rewrite() for facets", and Greg Miller kept only isCacheable "for the query implementations in LongRange and DoubleRange", so that discussion had the range queries in view. The LUCENE-10245 CHANGES entry says "DoubleRange and LongRange also support creating queries from these multi-valued sources" (lucene/CHANGES.txt:4438-4442). The direct overload builds a query equal to the documented fromSingleValued path's, so the two cannot be told apart at run time, and the code fix changes behaviour on that documented route too. The range counters do not rewrite either, the upstream test picks either overload at random (TestRangeFacetCounts.java:1703-1709), and a caller-side rewrite works.

Why the adjudication and the devil's-advocate verifier still read the direct overloads as a defect: the caveat sits on the fromSingleValued wrappers, not on getQuery(Query, DoubleValuesSource), whose javadoc states no restriction; the PR #543 exchange decided which methods the new Multi* classes have (they have no rewrite()), not that this Query may skip DoubleValuesSource#rewrite; the counters have no searcher, while this Query does, and core's rule is written for Queries; and in LUCENE-7736 Alan Woodward wrote "the only callers of rewrite are internal lucene classes" (while discussing equals() across readers), which suggests users were not expected to pre-rewrite. Verdict: a defect (adjudication, medium confidence), with intent contested: the devil's-advocate verifier called the case "not decisive", and the cross-model fact check rated it strong and would file this as a neutral design or documentation question.

One option was built and run in a scratch worktree of 645d4fe14a20; the other is not built. Which one fits depends on which contract the maintainers want the direct overloads to follow (see Why it is a bug); a maintainer could reasonably choose either, and the code fix is not the only reasonable answer.

The finder's code fix plus a javadoc update (4 files, +19/-10). Its two-line code comment was corrected on 2026-10-30: it now credits the Query sentence to DoubleValuesSource#rewrite only, and in LongRange cites the LongValuesSource class protocol. In DoubleRange.ValueSourceQuery.createWeight (and the same in LongRange with LongValuesSource), rewrite first and use the rewritten source for getValues and isCacheable. In MultiDoubleValuesSource / MultiLongValuesSource, "will also never be called" becomes "will also never be called when counting facets (the range queries from DoubleRange#getQuery do rewrite an unwrapped single-valued source)".

     public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float boost)
         throws IOException {
+      final DoubleValuesSource rewrittenSource = valueSource.rewrite(searcher);
       ...
-          final DoubleValues values = valueSource.getValues(context, null);
+          final DoubleValues values = rewrittenSource.getValues(context, null);
       ...
         public boolean isCacheable(LeafReaderContext ctx) {
-          return valueSource.isCacheable(ctx);
+          return rewrittenSource.isCacheable(ctx);
         }

Trade-offs

• Must land with LFV-770. toSortableLongDoubleValuesSource().rewrite() returns the plain cast adapter (DoubleValuesSource.java:193-195), which today never matters here. Probe: one document with value 2.5, LongRange over the sortable longs of 2.0 and 3.0, source fromDoubleField("v").toSortableLongDoubleValuesSource(): count 1 on main, 0 with this fix alone, 1 with LFV-770's one-line fix added (return inner.rewrite(searcher).toSortableLongDoubleValuesSource();).
• getQuery(Query, Multi*ValuesSource) unwraps fromSingleValued into the same query, so the fix also changes behaviour on that documented route: it starts rewriting there, against the Multi* note as written today; hence the javadoc update (or keep the two paths apart).
• One rewrite per createWeight, the same cost FunctionMatchQuery pays. A source the caller already rewrote is rewritten again: for fromQuery's rewritten form that returns the same object (WeightDoubleValuesSource.rewrite returns this), a legacy WrappedDoubleValuesSource makes a new wrapper (ValueSource.java:244-246), and core says implementations "should assume that this will only be called once" (DoubleValuesSource.java:94), so other pre-rewritten sources are not covered; FunctionMatchQuery behaves the same. No benchmark was run.
• Touches the isCacheable line that LFV-320 (the fast-match query's cacheability is ignored) also changes; coordinate the two.
• Score-dependent sources stay unsupported: the fixed query still calls getValues(context, null).

What the checks validated

• Scratch worktree of 645d4fe14a20, gradle, JDK 25: stock + test fails with assertions on and off; + fix: the test passes with assertions on and off.
• + fix: :lucene:facet:test --tests 'org.apache.lucene.facet.range.*' passes, 37 tests (TestRangeFacetCounts 22, TestDynamicRangeUtil 14, the repro 1); + fix and the test: :lucene:facet:check BUILD SUCCESSFUL (license, format, ecj lint and forbidden-API checks; :lucene:facet:test 307 tests, 7 skipped); :lucene:facet:renderJavadoc passes with no warnings.
• Earlier, code part shadowed onto the main jars: the finder's 3 tests and the independent reproduction's 7 pass; upstream TestRangeFacetCounts 22 of 22.
• With LFV-770's fix added (shadowed): the coupling probe and the minimal test pass.
• After the comment fix (2026-10-30), the new diff in a fresh scratch worktree: :lucene:facet:check passes again (307 tests, 7 skipped; format, ecj and forbidden-API checks) and the repro passes with assertions off; diff stats unchanged.
• Not run: the full Lucene suite, nightly tests, the sandbox module, benchmarks.

Add to both getQuery(Query, DoubleValuesSource / LongValuesSource) javadocs that a source which needs rewriting must be passed as source.rewrite(searcher).

Trade-offs

No behaviour change, it matches the Multi* note as written, and it has no dependency on LFV-770; but these stay the only Queries that take a caller-supplied values source and skip core's rule, the query is tied to one searcher for searcher-bound sources, and users must know about the silent QueryValueSource case themselves.

Method: a read-only cross-model critique, confirmed by execution (non-formal), a hunt unit (facet residual algorithms). The unit's bounded-exhaustive harnesses checked range drill-downs against counting, but only through the Multi* overloads with from*Field sources. Asked to find a concrete defect that those harnesses could not see, a reviewer from another model family (OpenAI GPT-5.6, reading only, running nothing) reported one finding at high confidence: the DoubleRange drill-down query never rewrites a rewrite-dependent DoubleValuesSource (DoubleRange.java:156-163, :190), with a one-document input (DoubleValuesSource.fromQuery(new MatchAllDocsQuery()), range [0, 2]), expected count 1 and a derived actual of "This DoubleValuesSource must be rewritten", plus the LongRange twin (LongRange.java:143-150, :177). The harness gap was its explanation of why the harnesses missed it. The finder read ValueSourceQuery, confirmed the defect with a public-API JUnit on the stock main jars (2026-10-27), and added the silent QueryValueSource variant through ValueSource#asDoubleValuesSource() and QueryValueSource. The independent reproduction added IndexReaderFunctions and the fast-match case; this doc's test is a one-method form of the finder's first test.