toSortableLongDoubleValuesSource() loses its sortable encoding when it is rewritten, so values and sorts use plain (long) casts

DoubleValuesSource#toSortableLongDoubleValuesSource() (public, final, in core since 9.12.0) is documented as "Convert to LongValuesSource by calling NumericUtils#doubleToSortableLong", and its values are exactly that as long as the source is not rewritten. Its rewrite(IndexSearcher) returns inner.rewrite(searcher).toLongValuesSource(), the plain (long)-cast adapter: the line is a copy of the cast adapter's own rewrite, where it is right. Both class javadocs tell clients to call rewrite before getValues, and IndexSearcher.search/searchAfter rewrite every Sort they are given, which rewrites the sort field's source. So on these paths every value is a cast: over 0.75, 0.25, -0.5 and 2.5 the rewritten source prints long(double(price)) and returns 0, 0, 0 and 2, and an ascending sort on its getSortField(false) returns docs 0, 1, 2, 3 with keys 0, 0, 0, 2 instead of docs 2, 1, 0, 3. Every key is then a (long) cast, which differs from the sortable long for every value except +0.0. The order can be wrong where the cast ties (fractions in one integer bucket, everything in (-1, 1) including -0.0) or saturates (values beyond the long range), because ties fall back to docID order; NaN is reversed outright: it casts to 0, so it sorts before every value >= 1 instead of last. No exception, nothing written. The reach is narrow: the only in-tree caller, the sandbox range-facet cutter, never rewrites, and no downstream use was found. The fix is one line, validated.

• DoubleValuesSource / LongValuesSource: produce one double (or long) per document, for example from a doc-values field. getValues(leaf, scores) returns the values for one segment.
• rewrite(IndexSearcher): "Return a LongValuesSource specialised for the given IndexSearcher". Both class javadocs make it the first step for clients: "clients should call rewrite(IndexSearcher) against the top-level searcher, and then getValues" (LongValuesSource.java:30-32, DoubleValuesSource.java:34-36).
• Two adapters from double to long. toLongValuesSource(): "Convert to a LongValuesSource by casting the double values to longs" (DoubleValuesSource.java:134-137), so 0.75 gives 0. toSortableLongDoubleValuesSource(): "Convert to LongValuesSource by calling NumericUtils#doubleToSortableLong" (:139-142), a bit-level encoding whose longs keep the order of Double.compareTo, NaN last (NumericUtils.java:38-43). It was added in 9.12.0 for the sandbox's range facets.
• getSortField(reverse): every LongValuesSource can be sorted on; the sort field holds the source (LongValuesSource.java:77-79).

1. Un-rewritten, the sortable source returns NumericUtils.doubleToSortableLong(in.doubleValue()) (DoubleValuesSource.java:159) and prints sortableLong(double(price)) (:188-190).
2. Its rewrite returns inner.rewrite(searcher).toLongValuesSource() (:192-195): a LongDoubleValuesSource, which returns (long) in.doubleValue() (:217) and prints long(double(price)). The line is identical to LongDoubleValuesSource.rewrite (:255-258), where re-applying toLongValuesSource() is right.
3. IndexSearcher.search(query, n, sort) and searchAfter rewrite the sort first (IndexSearcher.java:730, final Sort rewrittenSort = sort.rewrite(this);). LongValuesSortField.rewrite rewrites its source and keeps the old field only if the same object comes back (LongValuesSource.java:306-316). This rewrite always returns a new cast adapter, so a sort on this source through these methods always compares casts. A Sort used without them is not rewritten: search(Query, CollectorManager) (IndexSearcher.java:750) and TopFieldCollectorManager never call Sort.rewrite, and SortRescorer builds its collector from the Sort directly (SortRescorer.java:49); the same Sort in a hand-built TopFieldCollectorManager sorts correctly (executed).
4. toDoubleValuesSource() on it rewrites the inner source and re-wraps it (LongValuesSource.java:120-123), so FunctionScoreQuery (when scores are needed, FunctionScoreQuery.java:118-121), DoubleValuesSourceRescorer and FunctionMatchQuery, which rewrite their source, see casts too (the last from reading only).

Main a9779ef3c193, one segment, field price:

Ascending sort on src.getSortField(false): docs 0, 1, 2, 3 with keys 0, 0, 0, 2; expected docs 2, 1, 0, 3, which SortField("price", Type.DOUBLE) and DoubleValuesSource.fromDoubleField("price").getSortField(false) both return. The rewritten toDoubleValuesSource() view gives 0.0 for 0.75 instead of 4.604930618986332E18.

Edge values 2.0, -0.0, NaN, 1e300 (docs 0-3): after rewrite 2, 0, 0, Long.MAX_VALUE (expected 4611686018427387904, -1, 9221120237041090560, 9094988921128908188); the sort returns docs 1, 2, 0, 3, while Type.DOUBLE returns 1, 0, 3, 2. Eight values with searchAfter pages of 3: expected docs 1, 5, 4, 7, 2, 0, 6, 3, got 1, 3, 4, 5, 7, 0, 2, 6.

Precisely. After a rewrite every value changes encoding, integral ones too (2.0 gives 2 instead of 2^62); +0.0 is the only double whose cast equals its sortable long (both 0). So FieldDoc sort keys and comparisons with doubleToSortableLong bounds are wrong for every other value. Over non-NaN doubles the cast never gives a smaller value a larger key: it only merges values into ties (fractions that truncate to the same integer, everything in (-1, 1) including -0.0) or clips them (magnitudes beyond the long range). Ties fall back to docID order, which can put the larger value first (in the 8-value run, 1.5 in doc 0 before 1.25 in doc 2). NaN is reversed outright: doubleToSortableLong puts it after +Infinity, but it casts to 0, so it sorts before every value >= 1 (with doc 0 = 1.5 and doc 1 = NaN the sort returns docs 1, 0).

• Who is hit. Applications that call the public toSortableLongDoubleValuesSource() and then rewrite the result: by sorting on its getSortField(...) through IndexSearcher.search/searchAfter (which rewrite every Sort they are given), by following the documented rewrite-then-getValues protocol in their own code, or by passing toSortableLongDoubleValuesSource().toDoubleValuesSource() to FunctionScoreQuery (when scores are needed) or a DoubleValuesSourceRescorer.
• Default vs opt-in. No setting is involved: default IndexWriterConfig, codec and IndexSearcher; any index size (one document gives a wrong value). The opt-in part is the method itself.
• Silent and deterministic. On that path every value is a cast, every time; no exception, no log. The symptoms are the wrong values, keys, order and scores themselves; the source's toString also changes (sortableLong(...) becomes long(...)). With a top-n limit, a wrong order can also change which documents are returned; the set of documents that match the query does not change.
• Narrow reach. The only in-tree caller is the sandbox DoubleRangeFacetCutter (DoubleRangeFacetCutter.java:54), which never rewrites, so in-tree facet counts are fine. A GitHub code search found only copies of Lucene's sources (a port and a vendored copy, with the same line). Sorting doubles is usually done with DoubleValuesSource#getSortField, which is not affected. Scoring with the documented sortable-long values (about 4.6e18 for 0.75) would be an unusual use; affected versions score with the casts.
• Workaround. To sort doubles, use DoubleValuesSource#getSortField, which gives docs 2, 1, 0, 3 on the example. To read values, skipping the rewrite works only when the inner source needs none (a doc-values field does not; a fromQuery source throws until it is rewritten, DoubleValuesSource.java:768). The general workaround is to rewrite the double source and apply NumericUtils.doubleToSortableLong to its values, which is what the un-rewritten adapter does (:159).

# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
    => java.lang.AssertionError: value of 0.75 after rewrite, via long(double(price)) expected:<4604930618986332160> but was:<0>
       org.junit.Assert.fail(Assert.java:89)
       org.junit.Assert.failNotEquals(Assert.java:835)
       org.junit.Assert.assertEquals(Assert.java:647)
       org.apache.lucene.search.TestSortableLongValuesSourceRewrite.testRewriteKeepsSortableLongEncoding(TestSortableLongValuesSourceRewrite.java:45)

• An unconditional contract. DoubleValuesSource.java:139: "Convert to LongValuesSource by calling NumericUtils#doubleToSortableLong", on a public final method of a core class with no @lucene.experimental or @lucene.internal tag. rewrite is "Return a LongValuesSource specialised for the given IndexSearcher" (LongValuesSource.java:64-70): bound to a searcher, not a different function.
• Rewrite is part of the documented use. "To obtain a LongValues object for a leaf reader, clients should call rewrite(IndexSearcher) against the top-level searcher, and then getValues(LeafReaderContext, DoubleValues)" (LongValuesSource.java:30-32; the same in DoubleValuesSource.java:34-36). IndexSearcher.search/searchAfter rewrite every Sort they are given (IndexSearcher.java:730), so on that standard sort path the inherited, unrestricted getSortField can never deliver the documented encoding (only a Sort that bypasses those methods keeps it).
• The siblings keep their conversion. LongDoubleValuesSource.rewrite re-applies toLongValuesSource() (DoubleValuesSource.java:255-258), DoubleLongValuesSource.rewrite re-applies toDoubleValuesSource() (LongValuesSource.java:120-123). Only the sortable adapter switches, to a different class with a different toString and equals.
• Nothing says otherwise. The CHANGES entry (9.12.0, API Changes: "GITHUB#13568: Add DoubleValuesSource#toSortableLongDoubleValuesSource and MultiDoubleValuesSource#toSortableMultiLongValuesSource methods."), MIGRATE and the code comments say nothing about rewrite.
• Tests never covered it. No test in lucene/*/src/test calls toSortableLongDoubleValuesSource; the only caller never rewrites, which may be why the copied line went unnoticed.
• History. The class came in whole with #13568 ("Compute facets while collecting", @epotyom, merged by @gsmiller, 2024-08-12). None of its 415 review comments and 30 conversation comments mentions rewrite. The copied line comes from LUCENE-7736 (2017), which added rewrite to the values sources.

The strongest case for "intended": the method is plumbing for the experimental collect-time facets, and the facet module says rewrite is never called there. MultiDoubleValuesSource documents that DoubleValuesSource#rewrite(IndexSearcher) "will also never be called ... The faceting use-cases don't call rewrite or provide scores, which is why this simplification was made" (MultiDoubleValuesSource.java:43-47). Also, sorting doubles has a dedicated DoubleValuesSource#getSortField.

Why it fails: that note is in the class javadoc of the facet module's @lucene.experimental multi-valued wrappers and covers the sources they wrap. toSortableLongDoubleValuesSource() is a public final core method whose own class javadoc makes rewrite mandatory, whose javadoc promises the encoding without condition, and whose inherited sort field IndexSearcher.search rewrites automatically. An unrestricted public API that breaks on its documented path is a defect even if a better alternative exists. The weakest part of the case is the score path: NumericUtils says the integer form "is not usable" (NumericUtils.java:27-28), so a maintainer may discount sortable longs as scores; the protocol and Sort paths do not depend on that use. Verdict: not intended. The intended-lens cross-model review, the devil's advocate and the adjudication all rated the argument weak.

One fix was built and checked (A). B is a small variant; C is listed only to be rejected.

In SortableLongDoubleValuesSource.rewrite (DoubleValuesSource.java:192-195), return toSortableLongDoubleValuesSource() of the rewritten inner source instead of toLongValuesSource(). It mirrors the sibling adapters, which re-apply their own conversion.

@Override
public LongValuesSource rewrite(IndexSearcher searcher) throws IOException {
  return inner.rewrite(searcher).toSortableLongDoubleValuesSource(); // was: .toLongValuesSource()
}

Trade-offs

• Changes behaviour only for callers that rewrite this source; they get the documented values. Callers that never rewrite (the sandbox range-facet cutter) see no change.
• Allocation stays as today: rewrite returns a new object every time, as the cast adapter's rewrite does, so LongValuesSortField.rewrite still builds a new sort field.
• No benchmark was run. The work per value stays constant but changes: Double.doubleToLongBits plus a shift, an AND and an XOR (NumericUtils.java:48, :84-85) instead of a cast, the same work the un-rewritten source already does.

What the checks validated

• Scratch worktree at a9779ef3c193, gradle, seed DEADBEEF: stock, the repro fails 2 of 3 with -Ptests.asserts=true and =false; with the fix it passes 3 of 3 in both modes.
• Closest suites with the fix: core TestDoubleValuesSource, TestLongValuesSource, TestDoubleValuesSourceRescorer, TestSortRescorer, TestSort, TestSortOptimization; all of org.apache.lucene.queries.function.* (including TestFunctionScoreQuery, TestFunctionMatchQuery); org.apache.lucene.facet.range.*; org.apache.lucene.sandbox.facet.* (including TestRangeFacet, the only caller's tests): 30 upstream classes, 292 tests, 0 failures.
• ecjLint and checkGoogleJavaFormat for lucene/core with the test in place: pass.
• In the hunt: the same line shadowed in passes the finder's test and TestDoubleValuesSource + TestLongValuesSource (17 tests), makes 9.12.0 and 10.5.2 pass, and leaves the exhaustive harness with 0 mismatches (1,752 configs / 4.66 million checks; 272,344 configs / 1.30e9 checks).
• Not run: the full module suites.

DoubleValuesSource r = inner.rewrite(searcher); return r == inner ? this : r.toSortableLongDoubleValuesSource();

Trade-offs

Same values as A. For a searcher-independent inner source (a doc-values field), LongValuesSortField.rewrite and Sort.rewrite would then return the same objects (LongValuesSource.java:309-311, Sort.java:98). A small extra the sibling adapters do not have. Not run.

Javadoc only.

Trade-offs

Not recommended: the public sort field would stay useless on the standard path (IndexSearcher.search/searchAfter always rewrite it) and the method javadoc would contradict itself.

Method: bounded-exhaustive translation validation (the hunt's semi-formal tier), in a hunt unit on value sources against their documented semantics.

• The harness runs 103 DoubleValuesSource and 85 LongValuesSource cases on the stock jars (the core factories and adapters it models, including toLongValuesSource, toSortableLongDoubleValuesSource and toDoubleValuesSource compositions; not covered: vector-similarity sources, expression bindings, the facet module's multi-valued sources and TOP_SCORES block-max bounds) through rewrite(searcher) + getValues, FunctionScoreQuery, boostByValue/boostByQuery, getSortField and DoubleValuesSourceRescorer, and compares each result with an exact reference.
• Scope: every index of 1-2 documents over type-specific 12-value domains (doubles: absent, ±0.0, ±1, NaN, a negative NaN, ±Infinity, 2^63, 2^53+1, -0.5; longs include Long.MIN_VALUE/MAX_VALUE and ±(2^53+1)), all segmentations, deletion masks and two-index MultiReader splits: 1,752 configs, 4,658,632 checks, 649 mismatch keys, every one a source containing toSortableLongDoubleValuesSource() after rewrite; the un-rewritten values all matched; 0 with the fix shadowed.
• A larger run over the full bounded scope (every index of 1-3 documents over the full domains and of 4 documents over 5 values, 1-3 segments): 272,344 configs, 1.30e9 checks, the same 649 keys and nothing else; 0 with the fix.
• Found 2026-10-27; then checked by an independent reproducer, an intended-behaviour devil's advocate, reading-only reviews by a model from another family (OpenAI GPT-5.6), an adjudication and a credibility audit.