IntervalQuery equals/hashCode ignore pivot and exp, so BooleanQuery merges differently scored interval clauses

IntervalQuery can carry its own scoring curve: the public constructors (field, source, pivot) and (field, source, pivot, exp) set a saturation or sigmoid function that scores every matching document. IntervalQuery.equals and hashCode compare only the field and the intervals source, so two queries that differ only in pivot or exp are equal and have the same hash code.

BooleanQuery merges equal SHOULD clauses (with minimumNumberShouldMatch at most 1) and equal MUST clauses into one clause with boost 2, keeping the clause that was added first, and DisjunctionMaxQuery builds both of its weights from the first of two equal disjuncts. The second clause's curve is lost: on a one-word document, SHOULD(pivot 1, pivot 100) scores 1.0 instead of 0.509901, and SHOULD(pivot 100, pivot 1) scores 0.0198. Hit sets, counts and exclusions stay correct, but matching documents are scored with the first clause's curve, so scores are wrong and ranking can change, with default settings.

The scoring state came with LUCENE-8633 (8.0.0, 2019), which did not update equals/hashCode. Affected: 8.0.0 to 10.6.0 and main. Only applications that set pivot/exp themselves are hit: Lucene's query parser and the Elasticsearch, OpenSearch and Solr interval builders use the default curve only. Adding the score function to equals and hashCode fixes it.

1. The curve drives every score. The pivot constructor calls IntervalScoreFunction.saturationFunction(pivot) and the pivot/exp constructor sigmoidFunction(pivot, exp) (IntervalQuery.java:83-85, :97-99); the two-argument constructor uses saturationFunction(1) (:71-73). scorerSupplier hands scoreFunction to IntervalScorer (IntervalQuery.java:189-197), which scores every document with it (IntervalScorer.java:36-42, :50-53); the saturation score is weight * (1.0f - pivot / (pivot + freq)) (IntervalScoreFunction.java:72). explain uses it too (IntervalQuery.java:158-168).
2. Equality ignores it. equals ends in Objects.equals(field, that.field) && Objects.equals(intervalsSource, that.intervalsSource) and hashCode is Objects.hash(field, intervalsSource) (IntervalQuery.java:134-146). IntervalScoreFunction has value-based equals/hashCode on pivot (IntervalScoreFunction.java:90-101) and on pivot and exponent (:149-160), but nothing calls them. toString prints only field and source (IntervalQuery.java:116-119), so both queries print f:a.
3. BooleanQuery merges equal clauses and keeps the first. SHOULD and MUST clauses live in Multisets (BooleanQuery.java:133-135), a HashMap whose add of an equal element only bumps the count (Multiset.java:33, :80-84), so the clause added first is the surviving key (for clauses without a BoostQuery wrapper). rewrite de-duplicates SHOULD clauses when minimumNumberShouldMatch <= 1 (BooleanQuery.java:442-447) and MUST clauses (:450-455), and SHOULD clauses whose minimumNumberShouldMatch equals their count become MUST first (:570-588); deduplicateClauses merges them with boosts.merge(query, boost, Double::sum) (:726) and re-adds the survivor with boost computeQueryTermWeight(2) (:743-749), which is 2 for the default BM25Similarity. Result: (f:a)^2.0 scored with the first clause's curve.
4. DisjunctionMaxQuery does the same. Its disjuncts are a Multiset (DisjunctionMaxQuery.java:47) and it creates one weight per element (:115-117), so both weights come from the first disjunct; with tie-breaker 1 it rewrites to a SHOULD BooleanQuery (:273-279).

Adding scoreFunction to equals/hashCode, and nothing else, makes the repro and every shape below give the right value, so the equality is the cause.

DisjunctionMaxQuery([q1, q2], 0) gives 0.5, right only because q1 is the larger. The exponent constructor is hit the same way: on the document a b a, SHOULD(exp 1, exp 3) at pivot 4 scores 0.6666667 instead of 0.44444445. On a 4,000-document index with 100 colliding pairs, 218,304 of 221,120 matching-document scores differed from s1 + s2 for SHOULD(q1, q2), and the same count for MUST(q1, q2) (main ea57360912ed). Ranking changes once the query has another scoring clause: on the documents A = a b and B = a a a a a a a a a, SHOULD(q1, q2, an IntervalQuery on b) ranks B (1.8) above A (1.5) on stock main, and A (1.009901) above B (0.9825688) with the fix.

• Silent wrong scores, which can change ranking. Matching documents are scored with the first clause's curve (with two different pivots, every matching document gets a wrong score; sigmoid curves can agree at some frequencies), the result depends on clause order, and ranking changes once another scoring clause is present. No exception, no warning. Which documents match, count, MUST_NOT exclusions, deleted-document filtering and query-cache entries stay correct, because matching never reads the curve: the follow-up's probe ran 6,431,144 such checks with 0 failures on main, 10.5.2 and 9.12.3.
• Default settings, any size. One document, the default IndexWriterConfig, codec and similarity, a plain IndexSearcher. The test fails the same way with two seeds; the only randomized input, the class-level codec, does not affect interval scores.
• Who is exposed. Applications that call the pivot or pivot/exp constructors themselves and put two such queries, on the same field and an equal source and differing only in pivot/exp, into one BooleanQuery (SHOULD with minimumNumberShouldMatch at most 1 or equal to the SHOULD count, or MUST) or one DisjunctionMaxQuery; for example a proximity boost tuned at two pivots. Lucene has no production caller of these constructors, the flexible StandardQueryParser builds new IntervalQuery(field, source) (IntervalQueryNode.java:44), and a read-only code read found that Elasticsearch, OpenSearch and Solr also use only the two-argument constructor.
• Likelihood: rare. No public report of it was found in the searched sources since 8.0.0. The package "contains experimental classes" (package-info.java:21). The open issue #16820 proposes interval queries as the replacement for span queries, so their use may grow.

The test computes the two clause scores alone (0.5 and 0.0099) and asserts that the disjunction scores their sum. It uses ByteBuffersDirectory and new IndexWriterConfig(); Lucene's default similarity is a fixed BM25Similarity, and the test framework's class-level codec does not affect interval scores, so seeds DEADBEEF and 5EED1234 (5 iterations) fail the same way. The clause-order swap, MUST and DisjunctionMaxQuery shapes come from a separate probe and are not asserted, to keep one assertion.

The same file compiles and fails unchanged on the 9.1.0, 9.12.3, 10.5.2 and 10.6.0 jars; with only the LuceneTestCase import changed it fails the same way on 9.0.0 and 8.11.4, and with the package and import changed on 8.0.0; other 8.x releases were run only with the finder's test.

TestIntervalQueryDifferentPivotsDisjunction > testShouldClausesWithDifferentPivotsScoreAsSum FAILED
    java.lang.AssertionError: score of SHOULD(pivot 1, pivot 100) expected:<0.509901> but was:<1.0>
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:BC87C526B623306E]:0)
        at junit@4.13.2/org.junit.Assert.fail(Assert.java:89)
        at junit@4.13.2/org.junit.Assert.failNotEquals(Assert.java:835)
        at junit@4.13.2/org.junit.Assert.assertEquals(Assert.java:577)
        at org.apache.lucene.queries.intervals.TestIntervalQueryDifferentPivotsDisjunction.testShouldClausesWithDifferentPivotsScoreAsSum(TestIntervalQueryDifferentPivotsDisjunction.java:50)

• The curve is documented as changing the score. "The scores can be adjusted with two optional parameters", the pivot being "the sloppy frequency value at which the overall score of the document will equal 0.5" (IntervalQuery.java:41-51; constructors :75-99).
• BooleanQuery treats equal queries as score-interchangeable. deduplicateClauses merges "repeated queries into a single clause with a combined boost" (BooleanQuery.java:706-713; LUCENE-7925, "Collapse duplicate SHOULD or MUST clauses by summing up their boosts").
• Clause order changes the score of the same document: 1.0 vs 0.0198.
• History. equals/hashCode were written in LUCENE-8196 (2018), when field and source were the only state. LUCENE-8633 (#9679, 2019) added scoreFunction, the pivot/exp constructors and IntervalScoreFunction with value-based equals/hashCode, and did not touch IntervalQuery.equals. Its issue expects the scores to be "useful for things like proximity boosting on boolean queries"; equality was never discussed. Calling it an oversight is an inference; nobody said so.
• Core precedent. FeatureQuery compares its function (FeatureQuery.java:63-71), and FeatureField's SaturationFunction and SigmoidFunction, which IntervalScoreFunction mirrors, compare pivot and exponent (FeatureField.java:417-431, :503-509); two FeatureField saturation queries with pivots 1 and 100 are unequal and their disjunction sums correctly (executed). LUCENE-7808 (PayloadScoreQuery, SpanPayloadCheckQuery), LUCENE-10026 (CombinedFieldQuery: "ensures query rewrites don't drop CombinedFieldQuery clauses") and GITHUB#12260 (SynonymQuery) fixed the same kind of omission as bugs.
• Tests do not cover it. TestIntervalQuery.testEquality uses only the two-argument constructor (:475-493); testScoring uses the pivot constructors only in separate searches (:407-454).

The strongest case for "intended": Query#equals "is required so that QueryCache works properly", and "Typically a query will be equal to another only if it's an instance of the same class and its document-filtering properties are identical" (Query.java:95-105). Queries that differ only in pivot filter the same documents, so the current equals meets the literal text, and the query cache stores match sets, which stay correct; that cache rationale is a real argument, if not a sufficient one. The package is experimental; IntervalQuery itself builds default-scored copies new IntervalQuery(field, source) in MatchesIterator#getQuery (MatchesIterator.java:94, overridden in an anonymous FilterMatchesIterator in IntervalWeight.matches, IntervalQuery.java:179-183) and in the sources' visit(); and the query shape is unusual.

Why it fails: that sentence is a necessary condition for caching, not permission for a scoring rewrite to change scores, and BooleanQuery, a core consumer of equals, reads equality as "scores the same". No wording makes a disjunction's score depend on clause order. "Experimental" disclaims API stability, not correctness (LUCENE-10026's class was in the sandbox and was still fixed). Nothing compares the default-scored copies with user queries. Verdict: not intended, a defect (the independent reproduction, the cross-model review and the intended-behaviour check returned real; the adjudication confirmed it with high confidence). The defect is wrong, order-dependent scores from duplicate-clause merging, not a breach of a documented equals contract.

One fix was built and validated through gradle on 645d4fe14a20. It needs no API change, and equals/hashCode are identical in 10.6.0, so the same change applies to the 10.x line.

A 3-line change (+3/-2) in IntervalQuery.java. IntervalScoreFunction's SaturationFunction and SigmoidFunction already implement value-based equals/hashCode on pivot and exponent, so two queries built with the same pivot stay equal, and default queries (always saturationFunction(1)) stay equal to each other.

  @Override
  public boolean equals(Object o) {
    if (this == o) return true;
    if (o == null || getClass() != o.getClass()) return false;
    IntervalQuery that = (IntervalQuery) o;
    return Objects.equals(field, that.field)
        && Objects.equals(intervalsSource, that.intervalsSource)
        && Objects.equals(scoreFunction, that.scoreFunction);
  }

  @Override
  public int hashCode() {
    return Objects.hash(field, intervalsSource, scoreFunction);
  }

Trade-offs

• The default-scored copies that IntervalQuery builds in MatchesIterator#getQuery (the anonymous FilterMatchesIterator in IntervalWeight.matches, IntervalQuery.java:179-183) and in the sources' visit() (for example TermIntervalsSource.java:258) no longer equal a user query with a custom pivot. No in-tree code compares them; the highlighter suites pass.
• Queries that differ only in pivot/exp get separate query-cache entries (memory, not correctness), and a disjunction of them runs two scorers instead of one.
• saturationFunction(p) and sigmoidFunction(p, 1) give the same curve but are different classes whose equals checks getClass() (IntervalScoreFunction.java:90-96, :149-155), so with the fix the two queries compare unequal: safe (they are just not merged), but it can split cache entries more than needed.
• toString still prints only field and source, so the two clauses look the same when printed (cosmetic; see the add-on).

What the checks validated

• Gradle, scratch worktree of 645d4fe14a20, seed DEADBEEF: the repro fails on stock and passes with the fix, assertions on and off.
• org.apache.lucene.queries.intervals.* (including the repro): 118 tests, 0 failures.
• The whole :lucene:queries:test: 513 tests, 5 skipped, 0 failures.
• The other in-tree users of IntervalQuery: TestStandardQPEnhancements (queryparser, 23 tests) and TestMatchRegionRetriever, TestMatchHighlighter, TestUnifiedHighlighterTermIntervals (highlighter, 144 tests): 0 failures.
• checkGoogleJavaFormat, ecjLintMain, ecjLintTest, forbiddenApisMain and forbiddenApisTest of :lucene:queries pass with the fix and the repro test.
• Every probe shape gives the right value in both clause orders; earlier, the same change made the finder's tests, the independent reproduction's five tests, the intended-behaviour check's controls and 101 upstream intervals tests pass on main ea57360912ed.

Print a non-default pivot/exp in IntervalQuery.toString (IntervalQuery.java:116-119), so merged or rewritten queries are debuggable.

Trade-offs

Cosmetic and not needed for the fix; it changes printed query strings that applications or tests may compare.

Say in the javadoc that equal IntervalQuery objects may score differently.

Trade-offs

It would document clause-order-dependent scores for BooleanQuery and DisjunctionMaxQuery. Until a fix ships, keep two IntervalQuery objects that differ only in pivot/exp out of one BooleanQuery or DisjunctionMaxQuery; in a BooleanQuery, wrapping one in a BoostQuery does not help: deduplicateClauses unwraps boosts before merging (BooleanQuery.java:720-726), and the surviving curve then follows the Multiset's hash-map order, not clause order (SHOULD(new BoostQuery(q1, 2), q2) scores 0.029702961, 3 x q2's score, instead of 1.009901). In a DisjunctionMaxQuery a boost other than 1 keeps the disjuncts apart but doubles that disjunct's score, so it is no workaround either.

Method: semi-formal (a hunt unit, query object contracts).

• A bytecode scan of the stock main jars (JDK 25 ClassFile API) compared, for each of the 142 concrete Query classes, the fields read by createWeight, rewrite and its nested Weight classes with the fields read by equals/hashCode. It reported EQ-MISS IntervalQuery.scoreFunction, read by IntervalWeight.explain and scorerSupplier but not by equals.
• A bounded-exhaustive equality-congruence harness then compared every ordered pair within 28 small query families (249 queries, 3,510 pairs) on a fixed index, and witnessed equal IntervalQuery pairs with different per-document scores: pivot 1 vs pivot 10 vs pivot 1 with exp 2 (for example 0.6666666 vs 0.16666669 on the same document).
• The finder, an AI tool, wrote a 2-test public-API JUnit; the one-test repro was cut from it on 2026-10-29. The finder tagged the find "formal"; the registry records it as semi-formal because no model checker, solver or proof produced the signal.