FunctionRangeQuery over ValueSource.fromDoubleValuesSource compares in float, so double and long ranges silently gain or lose hits near a bound

ValueSource.fromDoubleValuesSource(DoubleValuesSource) adapts a DoubleValuesSource (a double or long doc-values field, LongValuesSource.toDoubleValuesSource(), a compiled expression) to the function-query ValueSource API. Its FunctionValues keeps the exact double in doubleVal but does not override getRangeScorer, so it inherits the generic default, which parses both bounds with Float.parseFloat and compares each value cast to float. FunctionRangeQuery, the only production caller of getRangeScorer, therefore range-filters the bridged source in float: any value that rounds to the same float as a bound counts as equal to it, and a bound or value too large for a float becomes infinite. Over epoch milliseconds [* TO 1700000000000] also returns an event one minute later (the window runs from t − 26,623 ms to t + 104,447 ms); [1000000000 TO 1000000000] matches 65 ids; [1.0 TO 1.0] matches 0.9999999999999999 and 1.0000000000000002; [* TO 1e300] matches 1e301. No error is raised. The same field through DoubleFieldSource / LongFieldSource, or FunctionMatchQuery over the same DoubleValuesSource, returns the exact hits. The bridge came in LUCENE-7737 (7.0.0) without a range override; it is affected from 7.0.0 on (executed on 11 versions up to main). Building the bridged values on DoubleDocValues, as the typed double sources do, fixes it (+3/−9 lines in ValueSource.java).

• The bridge: ValueSource.fromDoubleValuesSource(in) (ValueSource.java:267-272). It unwraps a source that came from ValueSource.asDoubleValuesSource() (since #15406, 10.4.0) and otherwise returns new FromDoubleValuesSource(in). It has no javadoc.
• FunctionRangeQuery: keeps its Number bounds as strings ("declared as strings because FunctionValues.getRangeScorer takes String args and parses them", FunctionRangeQuery.java:48-49) and asks the source's FunctionValues for a range scorer (:170-172), the only production caller of getRangeScorer.
• The float default: FunctionValues.getRangeScorer (FunctionValues.java:224-293). Typed values override it: DoubleDocValues compares doubles (DoubleDocValues.java:86-147), LongDocValues longs (LongDocValues.java:90-125).

1. FromDoubleValuesSource.getValues returns new FunctionValues() {...} (ValueSource.java:302) with floatVal = (float) inner.doubleValue() (:308-314), doubleVal = the exact inner.doubleValue() (:316-322) and exists = inner.advanceExact(doc) (:324-327), and no getRangeScorer.
2. So FunctionRangeQuery gets the default, which parses the bounds with Float.parseFloat (FunctionValues.java:245, :250) and compares floatVal(doc) (:261, :270, :279, :288) after exists(doc).
3. "1700000000000" becomes the float 1,700,000,038,912; t − 1, t and t + 60,000 all cast to that float, so <= holds for all three. "1.0E300" becomes Infinity, and (float) 1e301 is Infinity too.
4. DoubleFieldSource and LongFieldSource return typed values with their own range test, and FunctionMatchQuery applies its predicate to the exact double: over the same data they return the exact hits.

Window probe (every integer in a range indexed): [1000000000 TO 1000000000] matches 65 ids, 999,999,968 to 1,000,000,032 (the bound and 64 wrong ids); [t TO t] matches 131,071 timestamps, t − 26,623 to t + 104,447 ms (t and 131,070 wrong ones); [* TO t] admits values up to t + 104,447 ms. The hunt's independent reproducer also executed: {1.0 TO *] misses 1.0000000000000002 (exclusive bounds drop values that round onto the bound); [1e39 TO *] matches 3.5e38; at exact float midpoints both directions flip ([1.0000000596046448 TO 1.0000000596046448] misses the document with exactly that value, {1.0000001788139343 TO *] admits the equal one); an expressions-module source over-matches like a field. Not affected: documents without a value, integers below 2^24 (16,777,216), and bounds and values that are exact floats.

• Who. Code that combines FunctionRangeQuery (or its own getRangeScorer call) with ValueSource.fromDoubleValuesSource(...) of a DoubleValuesSource that did not start as a ValueSource: DoubleValuesSource.fromDoubleField/fromLongField, LongValuesSource.toDoubleValuesSource(), an expressions-module source. For expressions this is the only route into the ValueSource API since 7.0.0; up to 6.3, Expression.getValueSource returned values with an exact double range test (executed on 6.3.0), which LUCENE-7609 removed in 6.4.0.
• When. Deterministically, and only next to a bound: a value that differs from a bound but rounds to the same float counts as equal to it, so an inclusive bound admits it from just outside the range and an exclusive bound drops it from just inside; a value on the correct side of an inclusive bound is still judged right. The window is one float step wide (2^-24 to 2^-23 of the magnitude) around the bound's float: ±32 around 1e9; about 131 s around any epoch-ms bound from 2004 to 2039, placed differently per bound (t − 26,623 to t + 104,447 ms for t = 1.7e12). Numbers too large for a float (from about 3.4028236e38) become ±Infinity, so a bound and a value beyond it can compare equal; at exact float midpoints a value equal to a bound can round apart from it. Rare for ordinary doubles; ordinary for id- and timestamp-scale longs (65 ids match a point query near 1e9; at t = 1.7e12 a "published before t" cut-off admits items up to 104 s late).
• What goes wrong. A wrong hit set with no error: extra hits for inclusive bounds, missing hits for exclusive bounds, both at exact float midpoints. Query-time only; the index is not touched.
• Default vs opt-in. Default IndexWriterConfig, codec and searcher; the trigger is the API pair. No Lucene production code calls fromDoubleValuesSource, and FunctionRangeQuery is @lucene.experimental (FunctionRangeQuery.java:42). Since 10.4.0 (#15406) a typed ValueSource converted to a DoubleValuesSource and back is unwrapped and exact.
• Size of the check. A bounded-exhaustive sweep of about 30 range, exact and set query paths against an exact reference (983,164 double, 268,692 int and 483,557 long checks) found this path as the only one with unexplained mismatches on double, int and long fields (5,997, 606 and 1,528).

# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
    => java.lang.AssertionError: expected:<0> but was:<1>
       org.junit.Assert.fail(Assert.java:89)
       org.junit.Assert.failNotEquals(Assert.java:835)
       org.junit.Assert.assertEquals(Assert.java:647)
       org.junit.Assert.assertEquals(Assert.java:633)
       org.apache.lucene.queries.function.TestFromDoubleValuesSourceRange.testRangeComparesDoubles(TestFromDoubleValuesSourceRange.java:43)

• Two public paths over the same data disagree. FunctionRangeQuery over DoubleFieldSource("d") and FunctionMatchQuery(DoubleValuesSource.fromDoubleField("d"), v -> v >= 1.0 && v <= 1.0) return {1.0}; over fromDoubleValuesSource(DoubleValuesSource.fromDoubleField("d")) the same range returns three documents.
• The values are doubles. doubleVal is exact (ValueSource.java:316-322) and floatVal is a narrowing of it (:308-314). The comment above the default says a range "needs different behavior depending on the type of fields" (FunctionValues.java:228-229); the ValueFiller javadoc asks implementations to define their own "if their value is not a float" (:137-140).
• The typed double, long and int values override it. DoubleDocValues, LongDocValues and IntDocValues define their own getRangeScorer, as do DocTermsIndexDocValues (over term ordinals) and EnumFieldSource's values; FloatDocValues (whose values are floats), BoolDocValues and StrDocValues keep the float default. LUCENE-5247 ("expressions should use DoubleDocValues not FunctionValues directly", 9db7442d648, 2013, by rmuir) moved the typed range scorers into Int/Long/DoubleDocValues; its motivation was an UnsupportedOperationException, so it is the pattern, not a precision decision.
• Nothing in Lucene documents float semantics for the bridge. fromDoubleValuesSource has no javadoc; FunctionRangeQuery "matches docs in which the values in the value source match a configured range" (FunctionRangeQuery.java:32-33); upstream tests use the bridge only for scores, sorting and the round trip; no CHANGES or MIGRATE entry mentions range precision. Downstream reliance on today's float matching cannot be ruled out. The bridge is maintained API: #14654 and #15406 (2025) fixed other methods.

The strongest works-as-designed case (built by the intended-behaviour devil's advocate and a review by a model from another family, OpenAI GPT-5.6): FunctionValues is float-centric by heritage (the default parses floats, objectVal "by default return[s] a Float()", FunctionRangeQuery says "The score is the float value" and is @lucene.experimental); the adapter began in the LUCENE-7737 patch as a Solr-side helper that wrapped spatial distance sources for sorting and scoring, and that discussion called ValueSource "being phased out (or moved to Solr)"; no Lucene production code calls the bridge; FunctionMatchQuery is the modern exact alternative; and SOLR-6562 (2014, resolved Invalid) called float precision in Solr's sum/sub function math working as designed.

Why it fails: float scores do not imply float filtering; the bridged value is an exact double and only the range test narrows it (in SOLR-6562 the values themselves were computed in float); the typed double and long sources compare in their own type; nothing in Lucene documents the float behaviour; and results such as [* TO 1e300] matching 1e301, or an inclusive point query missing the document with exactly that value, are not "values between the specified range". A maintainer may still answer "use FunctionMatchQuery", or prefer the narrow fix B or a documented limitation; the reply is that this public adapter silently returns wrong hits and the fix is small and leaves the range test's per-document cost unchanged.

Two fixes were applied in a scratch worktree at a9779ef3c193 and run through gradle (seed DEADBEEF), with the repro and a regression test in TestFunctionRangeQuery; both pass. A is recommended; B is the narrow variant.

ValueSource.java only (+3/−9): import DoubleDocValues, replace new FunctionValues() with new DoubleDocValues(this) and drop the hand-written floatVal, which DoubleDocValues derives as (float) doubleVal. The range test is then DoubleDocValues.getRangeScorer (Double.parseDouble, doubleVal), as for DoubleFieldSource and, since LUCENE-5247, expressions.

 import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.queries.function.docvalues.DoubleDocValues;
 ...
-      return new FunctionValues() {
+      // DoubleDocValues compares ranges in double and derives floatVal etc. from doubleVal
+      return new DoubleDocValues(this) {
 ...
-        @Override
-        public float floatVal(int doc) throws IOException {
-          if (inner.advanceExact(doc) == false) {
-            return 0;
-          }
-          return (float) inner.doubleValue();
-        }

Trade-offs

• Scores, explain, toString(doc) and exists do not change (floatVal stays (float) doubleVal, 0 when missing); the range test does the same work per document as today (exists, then one value read).
• Float bounds over a float-valued source change (also with B): FunctionRangeQuery keeps 0.3f as "0.3", so over a FloatDocValuesField holding 0.3f, bridged with DoubleValuesSource.fromFloatField, [* TO 0.3f] and [0.3f TO 0.3f] match it today (as FloatFieldSource does) and not with the fix: the value is (double) 0.3f = 0.30000001192092896 > Double.parseDouble("0.3") (executed). FloatFieldSource stays the float-exact path. Name it in CHANGES.
• The other projections change from the FunctionValues defaults to DoubleDocValues: objectVal returns a Double (null when missing) instead of a Float; getValueFiller fills a MutableValueDouble with exists; byteVal, shortVal, intVal and longVal return casts, strVal returns Double.toString(doubleVal) and boolVal doubleVal != 0, where today they throw UnsupportedOperationException; bytesVal, which calls strVal, starts working; the values become an instanceof DoubleDocValues. A downstream that prints objectVal can see it.
• Cost: objectVal and the ValueFiller now call exists and doubleVal (advanceExact twice for a document with a value) where today they call floatVal once.
• Unchanged (read): documents without a value and NaN values never match; a missing exclusive bound still excludes an infinite value.
• ValueSource.java starts importing from queries.function.docvalues, which already imports from queries.function.

What the checks validated

• Stock a9779ef3c193: the repro fails 4 of 4 and the regression test fails (14 tests, 5 failed), with assertions on and off.
• Fix: 14 of 14 pass, assertions on and off; :lucene:queries:test passes, 517 tests, 5 skipped (the 512 existing tests plus the 5 new ones); ecjLint, checkGoogleJavaFormat and forbiddenApis pass.
• Classpath runner with the fix shadowed: the repro passes 4 of 4. Earlier in the hunt the same change passed every verifier's test, the history probe and the queries module.

The change is +22 lines in the anonymous class, from the hunt's independent reproducer: parse both bounds with Double.parseDouble, keep null as ±infinity, and compare doubleVal(doc) after exists(doc).

Trade-offs

The same Float-bound change as A (executed: 0.3f over a bridged float field no longer matches [* TO 0.3f]); no objectVal, ValueFiller or accessor change; the cost is a second copy of the comparison DoubleDocValues already has.

What the checks validated

• The repro passes 4 of 4 and TestFunctionRangeQuery 10 of 10, assertions on and off; :lucene:queries:test passes, 517 tests, 5 skipped; ecjLint, checkGoogleJavaFormat and forbiddenApis pass.
• Classpath runner with the reproducer's identical override shadowed: the repro passes 4 of 4.

regression-test.diff adds testRangeOverDoubleValuesSource: three documents with a double and a long value; [1.0 TO 1.0] must count 1, [* TO 1e300] 2, [* TO 1700000000000] over the long field 1. fix-with-test.diff is A plus this test.

Trade-offs

No upstream test range-filters a bridged source today. On stock main the test stops at its first assertion; it has no midpoint, exclusive-bound, missing-value or Float-bound case (the repro and the reproducer's tests cover the first three).

Change FunctionValues.getRangeScorer itself.

Trade-offs

FunctionValues.doubleVal throws UnsupportedOperationException by default (FunctionValues.java:60-62), so every float-only implementation that relies on the default would break.

Method: bounded-exhaustive cross-path translation validation on the stock jars (semi-formal tier), in a hunt unit on range-bound handling across query paths.

• The harness built every (lower, upper, includeLower, includeUpper) over a type-limit value alphabet, ran each through about 30 range, exact and set query paths and compared each hit set with an exact Java reference: 983,164 double, 268,692 int and 483,557 long checks; the float sweep (729,052 checks) had no unexplained class.
• The bridged path was added to the sweep after code reading showed that FromDoubleValuesSource has no range override. It left 5,997 double, 606 int and 1,528 long mismatches; every other mismatch was attributed to a registered or documented class.
• A Z3 floating-point model characterised when the float and double tests disagree. Its one assumption, parseFloat(Double.toString(b)) == (float) b, fails at exact float midpoints, which the independent reproducer then executed in both directions.
• The class was reduced to a public-API JUnit (found 2026-10-26 on main ea57360912ed), then checked by an independent reproducer, an intended-behaviour devil's advocate, reviews by a model from another family (OpenAI GPT-5.6), an adjudication, a history and prior-report follow-up and a credibility audit.