Sort fields built twice from one DoubleValuesSource or LongValuesSource compare unequal

DoubleValuesSource#getSortField and LongValuesSource#getSortField (and Expression#getSortField, which delegates to the first) build a new SortField around a new private comparator source on every call, and that comparator source has no equals or hashCode. SortField#equals compares comparator sources, and its javadoc requires a provided one to "properly implement equals (unless a singleton is always used)". So two sort fields built identically, even from the same source object, are unequal (their identity-based hash codes differed in every run that checked them), and so are Sorts built from them.

Sort order is not affected. The effect is on code that compares, deduplicates or caches on a Sort or SortField it builds anew for each request, such as a result cache keyed by query plus sort: it always misses for these sorts (reusing one Sort instance does hit). It has been so since 6.4.0. For expression sorts it is a regression: the ExpressionSortField they replaced had value equality, and the same expression sort built twice is equal on 6.3.0 (executed).

1. getSortField(reverse) returns new DoubleValuesSortField(this, reverse, 0) (DoubleValuesSource.java:111-113); the overload with a missing value does the same (:121-123).
2. The constructor hands SortField a new comparator source: super(producer.toString(), new DoubleValuesComparatorSource(producer, missingValue), reverse) (:633). DoubleValuesSortField (:628-661) overrides needsScores, toString and rewrite, not equals.
3. DoubleValuesComparatorSource (:667-703) holds producer and missingValue and declares neither equals nor hashCode, so two instances are equal only if they are the same object.
4. SortField#equals compares Objects.equals(this.comparatorSource, other.comparatorSource) (SortField.java:459) and hashCode hashes it (:469). Everything else it compares is the same for the two fields: field (producer.toString()), type CUSTOM, reverse flag, and SortField's own missing value, null because the missing value lives in the comparator source (SortField.java:195-200). Sort#equals and hashCode delegate to the fields (Sort.java:119, :125).
5. LongValuesSource has the same shape (LongValuesSource.java:77-79, :87-89, :289, :323-359). Expression#getSortField(bindings, reverse) is getDoubleValuesSource(bindings).getSortField(reverse) (Expression.java:107-109), and since #14654 (9.12.2, 10.2.2) ValueSource.fromDoubleValuesSource(dvs).getSortField returns in.getSortField(reverse) (ValueSource.java:332-334). Executed by the fact check: two such sort fields over fromDoubleField("price") are unequal on main and 9.12.2 and equal on 9.12.1, where that path still returned the queries module's own ValueSourceSortField, so #14654 also removed equality there.

The values sources must implement equals/hashCode themselves (abstract on both, DoubleValuesSource.java:125-129, LongValuesSource.java:55-59); the field sources used here are value-equal (two fromDoubleField("price") compare equal, probe), and the repro uses one source object for both calls anyway, so only the wrapper is at fault.

On main, with assertions on and off: a.equals(b) is false and the hash codes differ (identity-based), although field, type, reverse flag, missing value and toString (<double(price)>) are all equal; new Sort(a).equals(new Sort(b)) is false. A HashMap<Sort, TopDocs> over three identical requests (3 documents, prices 30, 10, 20) records miss, miss, miss and holds 3 entries; with the fix it records miss, hit, hit and holds 1. Both sort fields order the documents 10, 20, 30. Control: new SortField("price", SortField.Type.DOUBLE) built twice is equal, with equal hash codes.

• Who is hit. Code that compares, deduplicates, hashes or caches on Sort/SortField objects built by DoubleValuesSource#getSortField, LongValuesSource#getSortField, Expression#getSortField or (since 9.12.2 / 10.2.2) ValueSource.fromDoubleValuesSource(...).getSortField. The typical case is a search-result cache keyed by query plus sort, with the Sort built anew for each request: every lookup for such a sort misses, deterministically. Code that reuses one Sort instance, or keys its cache on its own request description, is not hit.
• Opt-in. The default relevance sort and plain field sorts (new SortField(field, type), SortedNumericSortField, ...) are not affected; function, expression and values-source sorts are.
• Seen downstream. On SOLR-18006 (2025-12-10) David Smiley noted that Solr's change to its distance sort "adds queryResultCache cache-ability since the prior code didn't implement equals/hashCode on all components of the SortField". The prior Solr code returned super.getSortField(true), which is DoubleValuesSource#getSortField (Solr's own values source has value equals). Solr worked around it in its own code (apache/solr#3923; SOLR-18006 lists fix versions 9.10.1, 9.11 and 10.0).
• What it does not do. Sort order is correct (checked on the probe's 3-document index), with assertions on and off. These fields cannot be index sorts (IndexWriterConfig#setIndexSort rejects a field without an index sorter, IndexWriterConfig.java:505-510). We found no non-test Lucene code that keys a map or set on Sort/SortField.
• Scale. Every lookup for an affected sort; the cost is lost cache hits and, for an unbounded map, one duplicate entry per request. Unreported in Lucene for about ten years.

The test sits in org.apache.lucene.search by upstream convention but uses only public API (DoubleValuesSource.fromDoubleField, LongValuesSource.fromLongField, getSortField(boolean)). Two methods, not one, because they are two separate fix sites and a failed assertEquals in one would hide the other. The same test is the suggested regression test: it passes with the fix below and passes the core lint checks as written.

Not asserted here (shown by the verification runs): hashCode, Sort equality, the HashMap<Sort, ?> miss, Expression#getSortField, and that sorting is correct.

# stock main 645d4fe14a20, gradle, seed DEADBEEF (same with -Ptests.asserts=false)
TestValuesSourceSortFieldEquals > testLongValuesSourceSortField FAILED
    java.lang.AssertionError: expected: org.apache.lucene.search.LongValuesSource$LongValuesSortField<<long(ts)>> but was: org.apache.lucene.search.LongValuesSource$LongValuesSortField<<long(ts)>>
        at org.apache.lucene.search.TestValuesSourceSortFieldEquals.testLongValuesSourceSortField(TestValuesSourceSortFieldEquals.java:31)
TestValuesSourceSortFieldEquals > testDoubleValuesSourceSortField FAILED
    java.lang.AssertionError: expected: org.apache.lucene.search.DoubleValuesSource$DoubleValuesSortField<<double(price)>> but was: org.apache.lucene.search.DoubleValuesSource$DoubleValuesSortField<<double(price)>>
        at org.apache.lucene.search.TestValuesSourceSortFieldEquals.testDoubleValuesSourceSortField(TestValuesSourceSortFieldEquals.java:26)
2 tests completed, 2 failed

# with the fix (asserts true and false)
BUILD SUCCESSFUL: 2 tests, 0 failures

• Contract on SortField (SortField.java:446-449): "Returns true if o is equal to this. If a FieldComparatorSource was provided, it must properly implement equals (unless a singleton is always used)." The same for hashCode (:463-466).
• Stated intent for values sources in sorts. LUCENE-7723 (7.0; CHANGES "DoubleValuesSource enforces implementation of equals() and hashCode()") gave the reason: "Given that we embed values sources in queries and sorts, from which we expect that they have working equals/hashCode, we should also implement equals/hashCode on LongValuesSource/DoubleValuesSource impls." The sort-field wrapper was not updated.
• Why SortField has equality. LUCENE-1481 (2008, Uwe Schindler): a query-result cache "uses the Sort/SortField as key into the cache". LUCENE-7798 (6.6) gave ToParentBlockJoinSortField equality because it "is going to be used as query result key". LUCENE-6946 (5.4.1, Bug Fixes) treated an incomplete SortField.equals as a bug.
• Regression for expressions. LUCENE-5220 (2013) gave ExpressionSortField value equality; LUCENE-7609 (6.4.0) deleted it and routed Expression#getSortField through DoubleValuesSource#getSortField. Release jars: equal on 6.3.0, unequal on 6.4.0. The ValueSource.fromDoubleValuesSource path lost equality the same way in 9.12.2 (#14654): equal on 9.12.1, unequal on 9.12.2 (executed by the fact check).
• Consistency. The ten other non-test SortField subclasses all compare by value. Nine override equals and hashCode: BinarySortField, SortedSetSortField, SortedNumericSortField, LatLonPointSortField, XYPointSortField, FeatureSortField, ToParentBlockJoinSortField, Geo3DPointSortField and Geo3DPointOutsideSortField. The tenth, ValueSource.ValueSourceSortField in the queries module (ValueSource.java:373-384), overrides neither but holds no comparator source until rewrite, so SortField's own comparison (field = the source's description, type, reverse) makes two identically built ones equal, with equal hash codes (executed by the fact check, assertions on and off).
• Tests. TestSort#testEquals asserts that separately built sorts are equal (TestSort.java:73-92). TestExpressionSortField#testEquals (:41-71) checks only inequality cases and assertEquals(sf1, sf1) on one instance; it dates from LUCENE-5220, so it stayed green through LUCENE-7609.

The strongest case for "intended": FieldComparatorSource is @lucene.experimental and its javadoc says nothing about equality; no Lucene code needs equal sorts; on LUCENE-7736 Adrien Grand (jpountz) accepted identity-only equality for some reader-bound values sources ("ok imo as long as a.equals(b) returns true if a == b"); on LUCENE-9451 they said "Reference equality is the right check"; and the expressions test asserts only inequality.

Why it fails: the requirement is written on SortField, which is not experimental, and the comparator source is Lucene's own, not the user's. The LUCENE-7736 bar is that a == b implies equal, and here both sort fields come from the same source object and are still unequal. The LUCENE-9451 remark is about rewrite's "did it change" check, not equals. The test gap is a missing positive case in a test written for a class that had value equality, and expression sorts regressed from 6.3.0 to 6.4.0. Verdict: not intended.

One option was built and validated in a scratch checkout of main 645d4fe14a20; the other two are alternatives with their trade-offs.

Add equals and hashCode to DoubleValuesSource.DoubleValuesComparatorSource and LongValuesSource.LongValuesComparatorSource (the Long one compares missingValue == other.missingValue). This is the object SortField's javadoc says must implement equals, and the only place that holds the missing value, so equality must include it. SortField, Sort and the sort-field classes stay unchanged.

// DoubleValuesSource.DoubleValuesComparatorSource
@Override
public boolean equals(Object obj) {
  return obj instanceof DoubleValuesComparatorSource other
      && Double.compare(missingValue, other.missingValue) == 0
      && producer.equals(other.producer);
}

@Override
public int hashCode() {
  return Objects.hash(producer, missingValue);
}

Trade-offs

• Equality is as good as the producer's equals. A values source with identity equality (accepted for reader-bound sources on LUCENE-7736) still makes two sort fields from the same source object equal, and sort fields over unequal sources stay unequal.
• For expressions this is the equality of the sources the bindings return (ExpressionValueSource compares them, ExpressionValueSource.java:146-171): custom bindings that return a new identity-equal source on each lookup keep Expression#getSortField unequal.
• Equal sort fields do not make a cached result valid for another index reader; a cache still has to be scoped to the reader, as with query keys.
• Double.compare treats 0.0 and -0.0 as different and NaN as equal to itself, consistent with Double.hashCode.
• Objects.hash boxes its arguments; this is not a per-document path.
• Not covered: the queries module's own ValueSourceSortField is equal before rewrite, and its rewrite returns a plain SortField around a new ValueSourceComparatorSource each time (ValueSource.java:379-383). Neither option changes that; it is outside this report.

What the checks validated

• Scratch worktree of 645d4fe14a20, gradle --offline, seed DEADBEEF. Stock: the repro fails with -Ptests.asserts=true and false. With the fix: the repro passes with both.
• :lucene:core:test --tests 'org.apache.lucene.search.*': 215 suites, 2,341 tests, 0 failures, with asserts true (49 skipped) and false (51 skipped).
• :lucene:expressions:test (170 tests, including TestExpressionSortField and TestExpressionSorts) and :lucene:queries:test --tests 'org.apache.lucene.queries.function.*' (111 tests, including TestFunctionQuerySort): 0 failures.
• Lint (checkGoogleJavaFormat, ecjLintMain, ecjLintTest, forbiddenApisMain, forbiddenApisTest, renderJavadoc on core, the repro test included): BUILD SUCCESSFUL.
• Not run: the rest of the core suite, other modules, x86.

Compare the producer, the reverse flag and the missing value in DoubleValuesSortField and LongValuesSortField. Not validated.

Trade-offs

Fixes SortField and Sort equality, but the comparator sources stay identity-equal (code that compares getComparatorSource() still sees unequal objects), the missing value must be read back through a cast, and it is more code than the recommended option.

Say in the getSortField javadoc that the returned sort fields are equal only to themselves. Not recommended.

Trade-offs

Keeps SortField's written contract broken for Lucene's own sort fields, keeps the 6.4.0 regression for expression sorts, and leaves every caching user to work around it, as Solr did.

Method: non-formal baseline, static analysis. In the study's static-analysis pass (2026-10-22, main 7cecf79523), SpotBugs reported EQ_DOESNT_OVERRIDE_EQUALS for the values-source sort-field classes.

• An independent verifier wrote a 9-test JUnit on the public API: 5 contract tests (same source twice, equal sources, Sort equality and hash, a HashMap<Sort, ?> lookup, Expression#getSortField twice) fail and 4 controls (plain SortField, SortedNumericSortField, FeatureField.newFeatureSort, LatLonDocValuesField.newDistanceSort are equal; different reverse, missing value or field stay unequal) pass, with several seeds.
• The same verifier ran a probe on the 6.3.0 and 6.4.0 release jars (expression sort equal, then unequal) and checked that value equals/hashCode on the two comparator sources makes all 9 tests pass.
• On 2026-10-29 the hunt cut the repro to one class with two one-line methods.