An Intervals.or inside maxwidth, fixField or ordered is not rewritten when it is one alternative of another or, so phrase, maxgaps and containedBy miss hits

Intervals.or promises to rewrite itself under a gap-sensitive parent (phrase, maxgaps, the big side of containedBy and containing) into one parent per alternative, because a disjunction drops any of its intervals that contains another one and a longer alternative would otherwise vanish (LUCENE-8477, 8.1.0). The rewrite asks an or for its alternatives only one level deep (Disjunctions.splitDisjunctions), so an or that sits inside maxwidth, fixField, extend, ordered or unordered, where that wrapper is itself one alternative of another or, is never split. With W = maxwidth(5, or("b c", b)), phrase(x, W, d) finds x b c d but phrase(x, or(W, y), d), with y in no document, finds nothing; the same holds for fixField, extend, ordered(or(a, "b a"), c), under maxgaps(0, ordered(...)), containedBy and containing, for the query string fn:phrase(x fn:or(fn:maxwidth(5 fn:or("b c" b)) y) d), and for analyzed text with a multi-word synonym. A plain nested or(or(...), y) is flattened and works. It reproduces on 8.1.0, 8.1.1, the latest patch of every minor line through 10.5.2 and the 10.6.0 intervals sources (4ae946e5a9b), and it is not fixed upstream. Expanding nested alternatives recursively in splitDisjunctions, with a guard against sources that list themselves, fixes every shape run and passes the queries, queryparser and highlighter test suites.

• Minimal intervals: an interval query returns position ranges per document. A disjunction returns only the intervals of its alternatives that contain no other of its intervals. In x b c d, or("b c", b) returns b [1,1] and drops b c [1,2], which contains it.
• The rewrite (LUCENE-8477, 8.1.0): a gap-sensitive parent (phrase, maxgaps, the big side of containedBy and containing) splits an or among its arguments into one parent per alternative. phrase(x, or("b c", b), d) becomes or(phrase(x, "b c", d), phrase(x, b, d)) and finds x b c d. The class javadoc promises this; or(false, ...) is the documented, lossy opt-out.
• Wrappers: maxwidth, fixField, extend, ordered and unordered do not split at construction. They hand their alternatives up when a parent asks (pullUpDisjunctions()).

1. phrase(...) asks Disjunctions.splitDisjunctions for the alternatives of each argument (BlockIntervalsSource.java:34, Disjunctions.java:39).
2. splitDisjunctions takes source.pullUpDisjunctions() once (Disjunctions.java:82). For or(W, y) that is [W, y] (DisjunctionIntervalsSource.java:146-152); W's own alternatives are never asked for, although MaxWidth.pullUpDisjunctions() would split them (FilteredIntervalsSource.java:72-74; likewise ExtendedIntervalsSource.java:134-143, FixedFieldIntervalsSource.java:59-68, OrderedIntervalsSource.java:78-80).
3. Two paths. (a) W has minExtent 1 (its alternative b), so it is grouped with y into one or (Disjunctions.java:83-92), against the comment's assumption that "all combinatorial sources have a minExtent > 1" (:78). (b) An alternative with minExtent 2, such as ordered(or(a, "b a"), c), gets its own branch but is still not split.
4. The query is built as BLOCK(x,or(MAXWIDTH/5(or(BLOCK(b,c),b)),y),d). At search time the inner or yields only b [1,1], the phrase needs d to start at position 2, but d is at 3, so it returns nothing.
5. A plain nested or(or("b c", b), y) works: DisjunctionIntervalsSource.simplify flattens a direct or child (DisjunctionIntervalsSource.java:58-68). Any non-or wrapper in between stops it, and so does an inner or(false, ...), which lists only itself, as the documented opt-out intends.

One-document indexes, default settings, main a9779ef3c193, assertions on and off. Every direct form finds 1 hit; every nested form, with an extra alternative that occurs nowhere, finds 0. With the recommended fix every nested form finds 1.

• Who. Users of the Intervals factories, of Intervals.analyzedText, and of the flexible StandardQueryParser fn: functions (from 9.1.0; fn: has or, maxwidth, extend, ordered, unordered, phrase, maxgaps and containedBy, but no fixField). No special settings (the probes use a plain IndexWriterConfig with the default codec and a plain IndexSearcher; the JUnit uses the test framework's randomized ones); any index size; assertions on or off. Nothing is written to the index, so a fixed Lucene fixes old indexes too.
• A realistic trigger. A query-time synonym york => "new york" (original kept) turns Intervals.analyzedText("york city", analyzer, "f", -1, true) into ORDERED(or(BLOCK(new,york),york),city) (IntervalBuilder.java:227 builds the or of synonym paths; :143-146 returns a bare ordered when maxGaps is -1). On in new york city, phrase(in, A) finds 1 hit, while phrase(in, or(A, boston)) and maxgaps(0, ordered(in, or(A, boston))) find 0. That is "a multi-word synonym inside one alternative of an any-of, under a zero-gap ordered parent".
• Silent. Fewer hits, no exception. Under a negation the miss could become an extra hit; that was not run.
• Rare. It needs an or whose short alternative can sit inside its long one, inside a non-or wrapper that is itself an alternative of another or, under a gap-sensitive parent. No report was found since it shipped in 2019. For such a query and document it misses every time.
• Not yet fixed. branch_10_6 4ae946e5a9b is affected, so the released 10.6.0 has it.

# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
    => java.lang.AssertionError: f:BLOCK(x,or(MAXWIDTH/5(or(BLOCK(b,c),b)),y),d) on 'x b c d' expected:<1> but was:<0>
       org.junit.Assert.fail(Assert.java:89)
       org.junit.Assert.failNotEquals(Assert.java:835)
       org.junit.Assert.assertEquals(Assert.java:647)
       org.apache.lucene.queries.intervals.TestPhraseOverOrOfWrappedOr.testPhraseRewritesOrInsideMaxWidthInsideOr(TestPhraseOverOrOfWrappedOr.java:46)

• Contract. The Intervals class javadoc (Intervals.java:47-54): gap-sensitive sources "will rewrite their sub-sources so that disjunctions of different lengths are pulled up to the top of the interval tree", with or(boolean, ...) for those who "care more about speed than about accuracy". Intervals.or (:121): "Automatically rewrites if wrapped by an interval source that is sensitive to internal gaps". or(false, ...) (:130-131): "can miss valid hits due to minimization". CHANGES 8.1.0, LUCENE-8477: "Interval disjunctions could miss valid hits if some of the clauses of the disjunction are minimized away. We now rewrite intervals ...".
• Tests. TestDisjunctionRewrites#testNestedMaxWidth (:138-154) and #testNestedFixField (:156-166) assert the rewrite through maxwidth and fixField; no upstream test puts such a wrapper inside another or. The repro's testNestedMaxWidthInsideOr is testNestedMaxWidth's query plus one outer alternative, and it loses its hit.
• Consistency. The nested query with the default rewrite behaves like the documented-lossy or(false, W, y): both 0 hits, while phrase(x, W, d) finds 1.
• Maintainers. On LUCENE-8477 (#9523), romseygeek gave the reason to rewrite inside the library: "disjunctions can end up being wrapped multiple times". On LUCENE-9204 (#10244): "Intervals automatically rewrite disjunctions (and have done since LUCENE-8477) so that queries behave as your or I would expect them to" (romseygeek) and "it's the key for correctness ... If it's not correct, it's a bug, not a feature" (jimczi).

The strongest case for "intended": the comment above splitDisjunctions (Disjunctions.java:75-78) describes a one-level split; before the LUCENE-8477 PR jimczi suggested documenting "other pathological cases due to minimum interval semantics" instead of rewriting; with minimal-interval semantics W alone only ever yields one-position intervals; the Expert hook pullUpDisjunctions() promises nothing about depth (IntervalsSource.java:66-71); and the package is marked experimental.

Why it fails: the comment states an assumption ("all combinatorial sources have a minExtent > 1") that these wrappers break, and path (b), with minExtent 2, fails without any grouping, so it does not describe a deliberate limit. The project chose to rewrite by default and to document or(false, ...) as the lossy option; nothing documents a depth limit, users rely on the Intervals javadoc rather than the Expert hook, and the built-in wrappers already answer correctly when asked (whether to recurse into custom hooks is a maintainer decision). Minimization is exactly what the rewrite exists to work around, and the direct form is rewritten for that reason. "Experimental" is an API-stability label, not permission for wrong results. Verdict: not intended, a defect.

Option A was built, formatted and tested in a scratch worktree of a9779ef3c193; option B is the finder's sketch, tested with the standalone runner; C is untested; D is a workaround for users.

The change is +29/-1 lines in Disjunctions.java, one private method. Before the existing minExtent grouping, an alternative whose own pullUpDisjunctions() returns more than one source is replaced by those sources, recursively. A source already being expanded (one that lists itself or an ancestor among its disjuncts) is kept whole. The nested form then builds the direct form's branches, with y added to the one-position branch. Sources that return one disjunct (terms, phrases, or(false, ...), atLeast) are untouched, so the opt-out stays lossy.

List<IntervalsSource> disjuncts = new ArrayList<>();
expandDisjunctions(source, source.pullUpDisjunctions(), new HashSet<>(), disjuncts);
for (IntervalsSource disj : disjuncts) { ... }   // the existing minExtent grouping

private static void expandDisjunctions(IntervalsSource source,
    Collection<IntervalsSource> disjuncts, Set<IntervalsSource> expanding,
    List<IntervalsSource> out) {
  expanding.add(source);
  for (IntervalsSource disj : disjuncts) {
    Collection<IntervalsSource> inner =
        expanding.contains(disj) ? Collections.emptyList() : disj.pullUpDisjunctions();
    if (inner.size() > 1) {
      expandDisjunctions(disj, inner, expanding, out);
    } else {
      out.add(disj);
    }
  }
  expanding.remove(source);
}

Trade-offs

• Why the guard. pullUpDisjunctions() is a public Expert hook. LFV-809's candidate fix returns this plus the alternative sequences for a whole unordered(X, X). With that fix and the unguarded option B, phrase(z, unordered(X, X)) throws StackOverflowError; with option A it finds its hit, and LFV-809's own tests pass with both (so they would not catch it). The guard compares with equals; a custom source that invents new unequal sources forever would still recurse. The hook's javadoc says nothing about depth, so recursing into custom implementations at all is a question for maintainers.
• Cost. More branches only for the affected nested shapes. Where the split runs in the list overload of Disjunctions.pullUp (phrase, overlapping, ordered/unordered when asked), the clause guard (Disjunctions.java:43-45) checks the product and nested alternatives now count toward it, so a query near the limit can now throw "Too many disjunctions to expand". The single-source overload (:66-73: maxwidth, the big side of containedBy/containing, notContainedBy/notContaining) has no clause guard, today or with the fix (read, not run). No benchmark was run.

What the checks validated

• Scratch worktree of a9779ef3c193, gradle, JDK 25: :lucene:queries:check :lucene:queryparser:check -x test (format, ecjLint, forbiddenApis) passes; the formatter leaves the fix unchanged.
• The 6-test repro passes 6 of 6 and the query-text test 1 of 1 (on stock: 5 of 6 and 1 of 1 fail).
• :lucene:queries:test :lucene:queryparser:test :lucene:highlighter:test (seed DEADBEEF, assertions on): 0 failures; queries 52 suites, 518 tests (123 in the intervals package, TestDisjunctionRewrites 12 of 12), 5 skipped; queryparser 31 suites, 534 tests, 1 skipped; highlighter 34 suites, 722 tests.
• :lucene:queries:test (seed 0D15C0DE, assertions off): 52 suites, 518 tests, 0 failures.
• The synonym, scope and shape probes give 1 hit for every nested form; the finder's tests, the independent reproducer's and the audit's tests pass and the devil's-advocate probe matches, as with option B. With LFV-809's candidate fix also applied, every probe line matches and LFV-809's tests pass.

The change is +18/-1 lines. Passes the finder's 4 tests, the fn: test, 117 upstream interval tests and 23 query-parser tests (standalone runner), the independent verifiers' tests, and this doc's 6 tests.

Trade-offs

Shorter, but it recurses without end when a source lists itself among several disjuncts: executed with LFV-809's candidate fix, StackOverflowError for phrase(z, unordered(X, X)).

DisjunctionIntervalsSource.simplify (:58-68) could replace any sub-source whose pullUpDisjunctions() has more than one entry, as it already does for nested ors.

Trade-offs

Changes every rewriting or tree, also outside gap-sensitive parents (toString, equals, cache keys, matches and highlighting), needs the same guard, and reverses the wrappers' "do not pull up at construction time" design.

Distribute by hand so that each wrapper sits directly under the gap-sensitive parent: or(phrase(x, W, d), phrase(x, y, d)) instead of phrase(x, or(W, y), d); or use a plain nested or without the wrapper where the wrapper is not needed.

Trade-offs

The application must know which alternatives contain an or; analyzed text hides it.

Method: reference-free translation validation of the real jar against algebraic laws (the hunt's "semi-formal" tier).

• A hunt unit on interval relations wrote a harness (ComplementLawsTV) that runs real interval sources on an index of 363 short documents and checks laws that need no reference semantics: each minimal interval of X is returned by exactly one of a relation and its negation, containedBy is a subset of overlapping, and or is commutative and idempotent. Every known interval fix was shadowed in, so that known bugs would not show up.
• A run of 30,000 random depth-3 pairs (24,510 checked) left one class it could not attribute (within0:neither:OTHER, 142 rows). The finder reduced it by hand to containedBy over or(fixField(g, or(BLOCK(a, a), z)), a), generalised it with a probe to phrase, maxwidth and extend, and wrote a one-document public-API JUnit on the stock jar.
• Independent AI tools then reproduced, reviewed and judged it (see the checks).