PatienceKnnVectorQuery over a SeededKnnVectorQuery modifies itself on every search, and overlapping searches of one query on two readers fail

PatienceKnnVectorQuery (early termination for HNSW search) can wrap a SeededKnnVectorQuery (HNSW search started from the hits of a seed query); both are @lucene.experimental. A seeded query needs a seed Weight built for the current searcher. SeededKnnVectorQuery.rewrite builds it into a new query, but PatienceKnnVectorQuery.rewrite assigns a new SeededKnnVectorQuery holding it to its own non-final delegate field (PatienceKnnVectorQuery.java:45, :296-312 on main 645d4fe14a20). Because equals/hashCode include that field and the seed Weight, one search changes the query's identity, against Weight's contract that "searching does not modify a Query, so that a Query instance can be reused" (Weight.java:29-32). The field is re-read for every segment, so when one query object is searched on two different readers at overlapping times, a search runs the other reader's seed Weight and fails with an AssertionError (assertions on) or, in the run with a 2-segment and a 1-segment reader, an ArrayIndexOutOfBoundsException (off). Sequential reuse still returns correct hits. #14688 introduced it in 10.3.0 while fixing an NPE on this path; every release from 10.3.0 to 10.6.0 and main are affected. Building the seed Weight into a new query, as SeededKnnVectorQuery.rewrite does, fixes it; the closest kNN suites and the whole core module pass with the fix. It rewrites into a plain PatienceKnnVectorQuery, so a subclass's overrides are skipped for a seeded delegate, and it uses a caller-supplied seed Weight as given (see the fixes).

1. The field. PatienceKnnVectorQuery.java:45: private AbstractKnnVectorQuery delegate;, the class's only non-final field (patience and saturationThreshold at :43-44 are final).
2. The assignment. rewrite (:296-312): if delegate is a SeededKnnVectorQuery, :301-309 assigns delegate = new SeededKnnVectorQuery(seededKnnVectorQuery.delegate, seededKnnVectorQuery.seed, seededKnnVectorQuery.createSeedWeight(indexSearcher), delegate.field, delegate.k, delegate.filter, delegate.searchStrategy), then super.rewrite runs the per-segment kNN search on this and returns a DocAndScoreQuery. The caller's object keeps the new delegate. createSeedWeight (SeededKnnVectorQuery.java:136-146) rewrites seed MUST + FieldExistsQuery(field) FILTER (plus the filter) and calls indexSearcher.createWeight(...), so the stored Weight belongs to this searcher's top-level reader.
3. Identity. PatienceKnnVectorQuery.equals/hashCode include delegate (:223-237); SeededKnnVectorQuery.equals/hashCode include seedWeight (SeededKnnVectorQuery.java:181-195); Weight keeps Object identity. So after any search the query no longer equals a fresh twin, and its hashCode includes the new Weight's identity hash: in every recorded run it differed from the twin's after one search and changed again after a second one.
4. Re-read per segment. approximateSearch (:196-204) calls delegate.approximateSearch(...) for each segment; SeededKnnVectorQuery.approximateSearch (:148-157) wraps the collector manager, and SeededCollectorManager.newCollector runs seedWeight.bulkScorer(ctx) on the caller's segment (:311-321). If another thread's rewrite replaced the field in between, that is the other searcher's Weight: TermQuery's check that the Weight was built for this top-level reader (TermQuery.java:120-122) fails under -ea; under -da, TermStates.get indexes the other reader's per-segment states with this segment's ord (TermStates.java:178-181): Index 1 out of bounds for length 1 when reader B has one segment (the run). The TermQuery path is this data's: every document has a vector, so the seed rewrites to the bare TermQuery; with other data or a kNN filter the seed Weight can be a BooleanQuery's and the failure can look different (read, not run).
5. The sibling. SeededKnnVectorQuery.rewrite (:119-134): when seedWeight is null it builds new SeededKnnVectorQuery(..., createSeedWeight(indexSearcher), ...) and returns rewritten.rewrite(indexSearcher); the original is untouched.

• One search: toString of the delegate goes from seedWeight=null to seedWeight=weight(id:0); equals with a fresh twin turns false; hashCode changed (in every recorded run, and again after a second search).
• A HashMap that stored a value under the query before the search no longer finds it afterwards (get returns null while size() is still 1).
• A BooleanQuery with the query as a clause, whose hashCode was computed before the search, searched once, fails assert hashCode == computeHashCode() (BooleanQuery.java:770) under -ea; an unseeded Patience clause passes.
• Searching the same object on reader A, then B, then A again returns the same top 5 as fresh queries each time.

• Who. Code that builds PatienceKnnVectorQuery.fromSeededQuery(...) or new PatienceKnnVectorQuery(SeededKnnVectorQuery, ...). Solr's DenseVectorField builds this combination when a request asks for a seed query and early termination, and its normal request parsing builds a fresh query object per request (read; Solr's caches and autowarming were not traced). Default IndexWriterConfig and IndexSearcher; one document is enough.
• On every search (deterministic). The query's equals/hashCode change. That matters to code relying on query immutability: maps or caches keyed on the query object (a HashMap lookup executed), and composite queries that cache a hash computed before the search (BooleanQuery asserts under -ea; with -da its cached hash silently goes stale, not asserted).
• Under overlapping searches on two different readers. One request fails with an AssertionError (-ea) or an ArrayIndexOutOfBoundsException (-da). Ways to get there: one cached or constant query object run in parallel against several indexes or shards in one JVM, or served by the old and the new searcher at once around an NRT refresh.
• Not claimed. Lucene's own LRUQueryCache is not affected: kNN queries always rewrite to DocAndScoreQuery or MatchNoDocsQuery before caching. Sequential reuse on different readers returns correct hits (executed). Concurrent searches on the same reader should not fail (read, not executed); if their two searchers had different similarities, the seed could be scored with the other searcher's (a TermWeight keeps its searcher's Similarity, TermQuery.java:43-58), which can change the HNSW entry points (read, not executed). The assertions-off runs used a 2-segment search A against a 1-segment search B; readers where the other one has as many segments or more were not run, so there is no silent-wrong-results claim. A searched query keeps the last search's seed Weight until the next rewrite; its size was not measured.
• Likelihood: rare. The identity change is constant but hurts only callers that depend on it; the crash needs one object shared by overlapping searches on different readers.

The test sits in org.apache.lucene.search, as Lucene's own tests do, but uses only public API: PatienceKnnVectorQuery.fromSeededQuery, SeededKnnVectorQuery.fromFloatQuery, a plain IndexSearcher (not newSearcher), ByteBuffersDirectory and the default IndexWriterConfig. The mutation happens in rewrite, before any per-segment work; the runs used Lucene's normal randomized test environment (seed DEADBEEF for this test).

It shows the root defect only. The crash needs two threads searching one query object on two different readers at overlapping times; the study reproduced it with a latch-driven test (a QueryTimeout that never times out is used only as a pause hook), which fails on stock main and passes with the fix. The printed weight(id:0) and the TermQuery frames below come from this data (every document has a vector, so the seed rewrites to the bare TermQuery).

It compiles against 10.3.0 through 10.6.0 and main. On 10.2.x it fails earlier with a different, already fixed NPE; older releases have no such API.

cp $REPRO/TestPatienceSeededQueryUnchangedBySearch.java $LUCENE/lucene/core/src/test/org/apache/lucene/search/
cd $LUCENE && ./gradlew :lucene:core:test --tests TestPatienceSeededQueryUnchangedBySearch -Ptests.seed=DEADBEEF -Ptests.asserts=true
# assertions off fails the same way: -Ptests.asserts=false

TestPatienceSeededQueryUnchangedBySearch > testSearchDoesNotModifyQuery FAILED
    java.lang.AssertionError: expected:<PatienceKnnVectorQuery{saturationThreshold=0.995, patience=7, delegate=SeededKnnVectorQuery{seed=id:0, seedWeight=null, delegate=KnnFloatVectorQuery:v[1.0,...][1]}}> but was:<PatienceKnnVectorQuery{saturationThreshold=0.995, patience=7, delegate=SeededKnnVectorQuery{seed=id:0, seedWeight=weight(id:0), delegate=KnnFloatVectorQuery:v[1.0,...][1]}}>
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:929F307F483DC7CE]:0)
        at org.junit.Assert.assertEquals(Assert.java:146)
        at org.apache.lucene.search.TestPatienceSeededQueryUnchangedBySearch.testSearchDoesNotModifyQuery(TestPatienceSeededQueryUnchangedBySearch.java:51)
> Task :lucene:core:test FAILED

# the same with -Ptests.asserts=false

# the study's two-reader test on the same stock build (supporting evidence, not the repro):
#   -ea: Caused by: java.lang.AssertionError: The top-reader used to create Weight is not the same as the current reader's top-reader
#          at org.apache.lucene.search.TermQuery$TermWeight.scorerSupplier(TermQuery.java:122)
#          at org.apache.lucene.search.SeededKnnVectorQuery$SeededCollectorManager.newCollector(SeededKnnVectorQuery.java:321)
#          at org.apache.lucene.search.PatienceKnnVectorQuery.approximateSearch(PatienceKnnVectorQuery.java:203)
#   -da: Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1
#          at org.apache.lucene.index.TermStates.get(TermStates.java:181)  (same path)

• Contract, current javadoc. Weight.java:29-32: "The purpose of Weight is to ensure searching does not modify a Query, so that a Query instance can be reused. ... IndexSearcher dependent state of the query should reside in the Weight." search/package-info.java:442-445: "Any IndexSearcher dependent state should be stored in the Weight implementation, not in the Query class."
• Core code relies on it. BooleanQuery caches its hash "since boolean queries are immutable" and asserts the cached value equals a recomputed one (BooleanQuery.java:760-772).
• The non-mutating shape exists next door. SeededKnnVectorQuery.rewrite (SeededKnnVectorQuery.java:119-134, from #14170) returns a new instance.
• No decision recorded. #14688 changed final AbstractKnnVectorQuery delegate into a mutable field only to create the seed Weight; its body ("This makes sure those queries are tested together and query rewriting works properly") and its one review ("LGTM") never mention mutation, reuse or thread safety. No upstream test checks a query after searching it: TestPatience{Float,Byte}VectorQuery.testToString asserts toString on a new query and then rewrites it (:96 / :106); with the randomly chosen seeded wrapper that rewrite runs the mutating path, but the test never looks at that object again, so the mutation goes unnoticed rather than being expected.
• Upstream treats this class of bug as a bug. #10322 (LUCENE-9282, open, type:bug): Surround parser queries change hashCode/equals on rewrite. #8041 (LUCENE-6984, fixed 2016): "queries should be considered immutable once created". #11467 (LUCENE-10431, open): Alan Woodward, "BooleanQuery is expecting all its subqueries to be immutable". On LUCENE-6369 (#7429) jpountz closed with "queries should now be immutable". Historical: lucene/CHANGES.txt:22476-22478 (1.3 RC1), "Queries are no longer modified during a search. This makes it possible, e.g., to reuse the same query instance with multiple indexes from multiple threads."

Strongest case: both classes are @lucene.experimental; Query.rewrite's javadoc (Query.java:72-83) never promises the query is left alone; sequential reuse still gives correct hits; so this is an internal detail plus a cache-key nit, and sharing one query object across concurrent searches on different readers is the caller's risk. The cross-model reviewers and the intended-behaviour verifier rated it weak.

Why it fails: the reuse guarantee is stated on Weight and in the package docs, not on rewrite, and it is what BooleanQuery's cached hash relies on; IndexSearcher is documented thread-safe (IndexSearcher.java:69-75), and neither class says its query is single-use or bound to a reader. @lucene.experimental waives API stability, not correctness, and the sibling class shows the non-mutating shape was available.

One fix was built and run in a scratch worktree of 645d4fe14a20 (the study's fix sketch); the alternatives were not built.

Make delegate final. In rewrite, when delegate is a SeededKnnVectorQuery whose seedWeight is null, build a new SeededKnnVectorQuery with createSeedWeight(indexSearcher) and return a new PatienceKnnVectorQuery around it, rewritten. The original query is never changed.

if (delegate instanceof SeededKnnVectorQuery seededKnnVectorQuery
    && seededKnnVectorQuery.seedWeight == null) {
  SeededKnnVectorQuery withWeight =
      new SeededKnnVectorQuery(
          seededKnnVectorQuery.delegate,
          seededKnnVectorQuery.seed,
          seededKnnVectorQuery.createSeedWeight(indexSearcher),
          delegate.field, delegate.k, delegate.filter, delegate.searchStrategy);
  return new PatienceKnnVectorQuery(
          withWeight, field, k, filter, searchStrategy, saturationThreshold, patience)
      .rewrite(indexSearcher);
}
return super.rewrite(indexSearcher);

Trade-offs

• Same pattern as the sibling, and final lets the compiler keep it that way.
• Caller-supplied seed Weight. A SeededKnnVectorQuery built with the public constructor that takes a seedWeight (#14838) is now used as given, as SeededKnnVectorQuery alone does; the stock code always replaced it. The caller must then build that Weight on the searcher that runs the query: one built on another reader fails the same way, and a TermWeight carries the Similarity of the searcher that built it (TermQuery.java:43-58). This path was not tested; add a test for it if you propose this fix. The next option keeps "always rebuild".
• Subclasses. It rewrites into a plain new PatienceKnnVectorQuery(...), so for a seeded delegate without a seed Weight a subclass's overrides (getKnnCollectorManager, approximateSearch, ...; possible since #14838 made the constructors public) are no longer used; the stock code ran super.rewrite on the subclass instance itself. Executed: a probe subclass overriding getKnnCollectorManager passes on stock and fails with this fix. SeededKnnVectorQuery.rewrite has the same limitation for its own subclasses. The weight-per-search option would avoid it (not built).
• Cost. One extra small query object per search, as in SeededKnnVectorQuery.rewrite. No public API change (delegate is private, the 7-argument constructor package-private).

What the checks validated

• Stock: the minimal test fails with assertions on and off; the study's two-reader test file (an identity test and the two-reader test) fails 2 of 2, on and off. With the fix: the minimal test passes 1/1 and that file 2/2, with assertions on and off.
• TestPatienceFloatVectorQuery, TestPatienceByteVectorQuery, TestSeededKnnFloatVectorQuery, TestSeededKnnByteVectorQuery, TestKnnFloatVectorQuery, TestKnnByteVectorQuery, TestKnnByteVectorQueryMMap and the new test: 8 classes, 278 tests, 0 failures, with assertions on and off.
• The whole core module (:lucene:core:test, assertions on): 748 classes, 8,897 tests, 0 failures, 322 skipped.
• checkGoogleJavaFormat, ecjLintMain, ecjLintTest, forbiddenApisMain and forbiddenApisTest pass (with the new test in place).
• Not covered: every run builds the seeded delegate without a seed Weight (fromSeededQuery / fromFloatQuery); the caller-supplied-Weight branch and subclasses were not tested (see the trade-offs).

Build the new PatienceKnnVectorQuery around a SeededKnnVectorQuery with a fresh seed Weight unconditionally, and run AbstractKnnVectorQuery.rewrite on that new instance through a private helper (calling .rewrite on it would loop).

Trade-offs

Keeps the stock behaviour for a caller-supplied seed Weight; slightly more code; not run. Like the recommended fix, it rewrites into a plain PatienceKnnVectorQuery, so a subclass's overrides are not used for a seeded delegate.

Create the seed Weight per search and pass it down through the per-search collector manager instead of storing it in a SeededKnnVectorQuery.

Trade-offs

Fixes the pattern in both classes, takes seedWeight out of equals and keeps rewrite on the caller's instance (so subclass overrides keep working), but reworks SeededKnnVectorQuery, whose constructors that take a Weight are public since #14838. Larger change; not run.

A javadoc note "not reusable, not thread-safe".

Trade-offs

Contradicts Weight.java:29-32 and BooleanQuery's immutability assumption, and leaves the crash in place.

Method: code reading, then TLA+ model checking and JUnit (the study's "non-formal" tier, because the first signal was code reading). In a hunt unit (kNN query layer, 2026-10-25), the finder noticed that rewrite assigns the query's own field. It then wrote SharedDelegate.tla (49 lines): two searches share the field, each assigns its own seed Weight and later uses the field. TLC found the invariant OwnWeightOnly (a search only uses the Weight it built) violated by a 4-state trace (A assigns, B assigns, A uses B's Weight), and no error with the fix (16 distinct states). The finder then bridged it to the stock jar with a public-API JUnit that fixes the interleaving with latches. The minimal one-search test of this doc was cut down from that JUnit's first method.