ParentsChildrenBlockJoinQuery's Weight#matches ignores the join, so parents and children of the wrong parents get Matches

ParentsChildrenBlockJoinQuery (new in 10.4.0, @lucene.experimental) returns the child documents that match childQuery and whose block parent matches parentQuery, at most childLimitPerParent per parent. Its scorer implements that join; its Weight#matches(context, doc) does not. It returns the union of parentWeight.matches(context, doc) and childWeight.matches(context, doc) for the same document and never checks the parent bit set, the document's own block parent or the limit. So matches() is non-null for documents the query does not match: a parent that matches parentQuery, a child that matches childQuery under a parent that does not match (or with no parent), and a child over the limit.

This bug does not change search hits, counts or scores (a separate advance() issue can, see the fix trade-offs), and the join's own hits get the right Matches. The damage shows wherever something asks Weight#matches about a document that is not a hit. As a MUST_NOT clause the spurious Matches make BooleanWeight#matches null for the enclosing query's real hits, so +text:size -join loses its Matches and the default UnifiedHighlighter its highlights; as a SHOULD clause it can satisfy minimumNumberShouldMatch on documents it does not match; Lucene Monitor's HighlightsMatch.MATCHER reports parents, children of non-matching parents, lone children and over-limit children as matches. The test framework's own CheckHits.checkMatches rejects +text:size -join and, on an index of two red-parent blocks, the join used alone. The method has been this way since the class was added (#14728); a block-local check in matches() (+30/-13, one method) fixes it.

• Blocks: a parent and its children are indexed together with IndexWriter#addDocuments, children first, parent last. A BitSetProducer (here QueryBitSetProducer(type:parent)) marks the parents.
• ParentsChildrenBlockJoinQuery(parentFilter, parentQuery, childQuery, childLimitPerParent) (10.4.0, experimental): "returns the matching child documents for matching parent documents indexed together in the same block", at most childLimitPerParent per parent (class javadoc, ParentsChildrenBlockJoinQuery.java:43-51 on main a9779ef3c193).
• Weight#matches(context, doc): for one document, returns Matches (which fields and positions matched) or null "if the document does not match the parent query", the parent query being the Weight's own query (Weight.java:61, :68-69). BooleanWeight#matches, the UnifiedHighlighter (on by default), Lucene Monitor's HighlightsMatch and the test framework's CheckHits rely on it.

1. The join lives in the scorer. ParentsChildrenBlockJoinScorer (:265-452) walks the parent and child iterators together: nextDoc() (:314-347) counts the children returned for the current parent (childDocCount against childLimitPerParent) and alignParentAndChildIterator() (:384-404) keeps the child inside the block of a matching parent, using the parent bit set. Parents are never returned.
2. matches() skips all of it. ParentsChildrenBlockJoinWeight#matches (:248-262) asks parentWeight.matches(context, doc) and childWeight.matches(context, doc) about the same doc id, returns null only when both are null ("Neither matches"), and otherwise returns MatchesUtils.fromSubMatches of the non-null ones ("Combine non-null matches"). Nothing looks at the parent bit set, the doc's block parent or the limit.
3. What the union does. For a valid parentQuery (it matches only parents; the scorer throws when it lands on a parentQuery match that is not a parent, validateParentDoc, :425-429) parentWeight.matches is null on every child, so on the join's own hits the union is just the child's Matches, which is right. Its only effect is "match" for non-hits: a parent that matches parentQuery (doc 2), a child that matches childQuery under a parent that does not match or with no parent in the segment (doc 3), a child over the limit (doc 1 with limit 1).
4. Who believes it. BooleanWeight#matches (BooleanWeight.java:134-164) returns null for a document when a prohibited clause's matches() is non-null (:142-146) and counts every non-null SHOULD clause toward minimumNumberShouldMatch (:153-162). The UnifiedHighlighter uses Weight#matches by default (UnifiedHighlighter.java:110, :518-527) and collects no offsets when it is null (FieldOffsetStrategy.java:143-150). Monitor's HighlightsMatch.MATCHER calls matches() for every document of the batch and records a match whenever it is non-null (HighlightsMatch.java:44-62).
5. The sibling does it right. ToParentBlockJoinQuery's weight advances its own scorer to the document and returns MATCH_WITH_NO_TERMS (ToParentBlockJoinQuery.java:200-219): "The default implementation would delegate to the joinQuery's Weight, which matches on children. We need to match on the parent instead".

• Highlighting. With the join as a MUST_NOT clause, the enclosing query's real hits get null Matches, so the default UnifiedHighlighter returns them unhighlighted (size large instead of <b>size</b> large; the withWeightMatches(false) control highlights) [executed on main, 10.5.2 and 10.4.0]. As a SHOULD clause under minimumNumberShouldMatch the join can be counted on documents it does not match [executed through Monitor].
• Lucene Monitor. HighlightsMatch.MATCHER differs from QueryMatch.SIMPLE_MATCHER in all 7 probe cases. Four use the stored join "children with body:size of acl:public parents": alone, with childLimitPerParent 1, and for one-document batches of a lone parent or a lone child. Three put body:size next to a join whose childQuery is type:child: of acl:public parents as a FILTER clause and as a SHOULD clause under minimumNumberShouldMatch 2, of acl:restricted parents as a MUST_NOT clause (true matches dropped) [executed on main, 10.5.2 and 10.4.0].
• Other Weight#matches callers. Code that asks about documents not known to be hits, for example to label which named clause matched (NamedMatches), gets the same wrong answer [read, not executed].
• Settings. No special setting is needed: a plain main() probe with the default IndexWriterConfig, the default codec and a plain IndexSearcher gives the same Weight#matches answers on main (assertions on and off), 10.4.0 and 10.5.2. The JUnit repros run under LuceneTestCase, which picks a randomized test codec (Asserting(Lucene104) in the recorded runs). Default highlighter flags. The mechanism does not depend on index size, but a visible effect needs a document that matches parentQuery or childQuery but not the join, and a caller that asks Weight#matches about it; the executed indexes have up to 5 documents.
• Not hit. Search hits, counts and scores (this bug does not touch the scorer; a separate advance() issue can return an over-limit hit in a conjunction, see the fix trade-offs); the Matches of the join's own hits; the join alone or as a required clause, highlighted on its hits (no visible symptom, although on some indexes CheckHits rejects the join alone).
• Reach. The class is @lucene.experimental, new in 10.4.0 and built programmatically (no query parser makes it); a code search found no use in Elasticsearch, OpenSearch or Solr. Likelihood: rare.

# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
    => java.lang.AssertionError: Matches of parent doc 1 expected null, but was:<org.apache.lucene.search.MatchesUtils$3@4e390179>
       org.junit.Assert.fail(Assert.java:89)
       org.junit.Assert.failNotNull(Assert.java:756)
       org.junit.Assert.assertNull(Assert.java:738)
       org.apache.lucene.search.join.TestParentsChildrenMatchesParentDoc.testParentDocHasNoMatches(TestParentsChildrenMatchesParentDoc.java:59)

• The contract. Weight.java:68-69: "Returns Matches for a specific document, or null if the document does not match the parent query", the parent query being the Weight's own query (@param query the parent query, :61). The class javadoc (ParentsChildrenBlockJoinQuery.java:43-51) defines the hits as matching children of matching parents, at most childLimitPerParent per parent.
• Lucene's own oracle. CheckHits.MatchesAsserter (CheckHits.java:646-688, run by QueryUtils.check) requires non-null Matches on each hit and, for each hit after the first one collected whose previous document was not a hit, null on that previous document (the message names the hit). It fails on +text:size -join ("Unexpected null Matches object in doc3") and, on an index of two red-parent blocks, on the join alone ("Unexpected non-null Matches object in non-matching doc2", about the red parent doc 1), while ToParentBlockJoinQuery passes on that index [executed].
• The sibling. ToParentBlockJoinQuery gives up child terms and returns MATCH_WITH_NO_TERMS from its own scorer rather than report Matches the join does not produce (LUCENE-8229, #9276; ToParentBlockJoinQuery.java:200-219).
• The rule, stated by the committer who merged #14728. In #15469 (comment of 2026-01-13): "Weight#scorerSupplier() and Weight#matches() are expected to be semantically the same". It was said about FilterWeight subclasses in general and names no class.
• Nothing accepts the union. No test, javadoc, CHANGES or MIGRATE entry describes it. The class's only test, TestParentsChildrenBlockJoinQuery, never calls matches(), CheckHits or QueryUtils.check; #14728's conversation and review comments and the issue #14565 never discuss matches().

The strongest case for "intended": the comment "Combine non-null matches" (:257) shows the union was written on purpose; the class and Weight#matches are @lucene.experimental; the union has a use, since with the default highlighter a join query also highlights the non-hit parent (<b>red</b> shirt) and keeps highlights on join hits that a correct matches() loses on the highlighter's one-document reader; and a correct matches() costs more and must not call this Weight's own scorerSupplier() twice per leaf (LFV-167).

Why it fails: the union is taken on the same doc id, so it never adds the parent's terms to a child hit; its only effect is answering "match" for non-hits, against the explicit null-for-non-match contract that BooleanWeight, Monitor and CheckHits depend on. Highlighting a parent the query does not match is not something the query promises, and Lucene already chose correctness over extra highlights in the sibling. "Experimental" allows API changes, not a method that contradicts its own javadoc. Every reviewer of the claim, including two from another model family, rated the intended-behaviour case weak.

One fix was applied in a scratch worktree at a9779ef3c193 and run through gradle (seed DEADBEEF); the finder's alternative was validated in the hunt and re-run here with its class shadowed; the third option was not run.

Replace the body of ParentsChildrenBlockJoinWeight#matches (+30/-13, also removing the three imports the union used). It mirrors the scorer's definition of a hit (the block parent matches parentQuery, the child matches childQuery and is among the first childLimitPerParent matching children of its block) and returns the child's own Matches for hits, so hits keep exactly the Matches they get today. It builds the parent and child scorers directly, as ToParentBlockJoinQuery#matches builds its own, so it does not use up this Weight's single scorerSupplier() call per leaf (LFV-167). Work per call: one parentFilter.getBitSet call (cached per segment by QueryBitSetProducer; a custom BitSetProducer need not cache), two fresh sub-scorers, one advance on each, a walk over the matching children of one block and, for a hit, childWeight.matches. Not benchmarked.

final BitSet parentBits = parentFilter.getBitSet(context);
if (parentBits == null || parentBits.get(doc)) {
  return null; // parent documents are never hits
}
final int parentDoc = parentBits.nextSetBit(doc);
if (parentDoc == DocIdSetIterator.NO_MORE_DOCS) {
  return null; // no parent after this doc
}
final Scorer parentScorer = parentWeight.scorer(context);
if (parentScorer == null || parentScorer.iterator().advance(parentDoc) != parentDoc) {
  return null; // the block's parent does not match parentQuery
}
final Scorer childScorer = childWeight.scorer(context);
if (childScorer == null) {
  return null;
}
final int firstChild = parentBits.prevSetBit(parentDoc - 1) + 1;
final DocIdSetIterator childIt = childScorer.iterator();
int matchingChildren = 0;
for (int child = childIt.advance(firstChild); child <= doc; child = childIt.nextDoc()) {
  if (++matchingChildren > childLimitPerParent) {
    return null; // over the per-parent limit
  }
  if (child == doc) {
    return childWeight.matches(context, doc);
  }
}
return null; // doc does not match childQuery

Trade-offs

• Known gap, a separate defect. In the hunt's bounded-exhaustive harness (every block layout of up to 4 documents), 40 of 108,336 checks of +text:a +join still differ with the fix, all with childLimitPerParent 1 or 2: the conjunction moves the join with advance(), which does not count the block's earlier children, so the search itself returns a child that nextDoc() order puts over the limit, and the fixed matches() says no. That lives in the scorer's advance() (ParentsChildrenBlockJoinQuery.java:350-382) and is not in the registry.
• Highlighting interplay. With any correct matches(), join hits whose parentQuery uses the highlighted field lose highlights that today's union produces by accident. The join is an opaque leaf to the highlighter (ParentsChildrenBlockJoinQuery.java:116-117), so a postings offset source falls back to analysis (UnifiedHighlighter.java:1253-1272); with analysis (and term vectors) the default UnifiedHighlighter evaluates Weight#matches on a reader whose highlighted field holds only the hit document, where the hit's parent cannot match. The POSTINGS_WITH_TERM_VECTORS source uses a different reader and was not tested. Hunt harness, highlighter oracle, every block layout of up to 3 documents on one segment, two highlighted fields: all 294 such checks per field pass on stock main and all 294 per field fail with this fix; the finder's fix gave the same, and 144 of 144 per field on two segments. That highlighter defect is registered separately as LFV-505 (novelty "known": a partial downstream overlap with elastic/elasticsearch#96068), established for the tested block joins whose condition uses the highlighted field.
• Nested joins [read, not executed]. It calls parentWeight.scorer(context) and childWeight.scorer(context) on every call. If a sub-query is or contains a ParentsChildrenBlockJoinQuery, that inner Weight's one-scorerSupplier()-per-leaf guard (:191-197, LFV-167) throws on the second outer matches() call for a leaf, or on the first one after the inner Weight already scored it; today's union calls only the sub-weights' matches(). Option B has the same exposure.
• No parent-query validation [read]. Like today's matches(), it does not repeat the scorer's validateParentDoc check (:425-429): with an invalid parentQuery that also matches children, the scorer may throw while matches() just answers for the block's real parent.
• Add a CheckHits.checkMatches (or QueryUtils.check) call to TestParentsChildrenBlockJoinQuery; no test of the class calls matches() today.

What the checks validated

• Scratch worktree at a9779ef3c193, gradle, seed DEADBEEF: the repro passes 4 of 4 with assertions on and off, together with TestParentsChildrenBlockJoinQuery (4 + 8 = 12 tests). In an earlier scratch worktree with the same fix, the whole lucene/join module with the finder's and the verifiers' tests added (11 tests; this doc's repro was not in that run): 115 tests, 3 skipped, 0 failures. checkGoogleJavaFormat, ecjLintMain and forbiddenApisMain pass (run after the added tests were removed, so on the production edit only).
• With the fixed class shadowed into the stock a9779ef3c193 jars: the highlighter test 1 of 1, the Monitor probe 7 of 7, the finder's 3, the independent reproducer's 6 and the devil's advocate's 2 tests pass.
• Hunt harness, every block layout of up to 4 documents: stock 44,808 of 108,336 checks fail for the join alone on one segment; with the fix 0 for the join alone, *:* -join and msm=2(join, text:b) on one and two segments, and the 40 advance() cases above.

Build a ParentsChildrenBlockJoinScorer from fresh parent and child scorers and call nextDoc() until it reaches doc; return the child's Matches only on a hit.

Trade-offs

Every call walks the segment up to doc, so a caller that asks about every document (Monitor's HighlightsMatch.MATCHER) does quadratic work per batch (not measured). It decides membership with nextDoc(), so it shares the advance() gap of option A, and it calls the sub-weights' scorer() like option A, with the same nested-join exposure [read]. Its diff leaves ArrayList, List and MatchesUtils imported but unused.

What the checks validated

In the hunt on ea57360912ed: the finder's, the highlighter, the reproducer's and the Monitor tests pass, and the join-module run gives the same result as without it (104 tests found, 101 successful, 3 aborted because the nightly test group was off, 0 failed). On a9779ef3c193, shadowed: the finder's tests 3 of 3 and the Monitor probe 7 of 7 pass.

Call scorer(context) and advance(doc), as the sibling does.

Trade-offs

Not viable as is: any call after this Weight has already supplied a scorer for the leaf (the second matches() call of a fresh Weight, or the first one after it scored the leaf) trips the one-scorerSupplier()-per-leaf guard (ParentsChildrenBlockJoinQuery.java:191-197, LFV-167), and advance() ignores the limit (the gap above). Not run.

Method: bounded-exhaustive translation validation on the stock jar (the semi-formal tier).

• First signal: a block-join hunt compared the real join jar with an independent brute-force join over every small index. Its Weight.matches check reported 3,512, 3,460 and 3,456 mismatches for childLimitPerParent 1, 2 and unlimited at up to 4 documents (about 1.3 million each at up to 7). It was recorded as an unpromoted lead; a second hunt on explain/score coherence noted the same defect as a lead.
• A later hunt on Matches coherence across module queries promoted it with its own harness: oracle M1, Weight#matches non-null exactly for the documents in a fresh scorer's match set, over every block layout of up to 5 documents with every text from {"", a, b, "a b"}, on 1 and 2 segments, for the query alone and inside +a +Q, *:* -Q and msm=2(Q, b). For example 446,600 of 1,091,376 checks failed for the join alone on one segment; ToParentBlockJoinQuery and the older ParentChildrenBlockJoinQuery had 0. The finder then wrote a public-API JUnit on the stock main jar.
• Found 2026-10-26 on main ea57360912ed; then checked by an independent reproducer, an intended-behaviour devil's advocate, reading-only reviews by a model from another family (OpenAI GPT-5.6), an adjudication, a security screen and a credibility audit.