DrillSideways with an ExecutorService wraps failures of its searches in RuntimeException(ExecutionException), unlike the sequential mode

DrillSideways runs a drill-down search plus one drill-sideways search per drilled dimension, either one after another on the caller's thread or, when an ExecutorService is passed to its 5-argument constructor, as parallel tasks. In the parallel mode, searchConcurrently collects the tasks with Future.get and turns any task failure into throw new RuntimeException(e) around the ExecutionException. So the caller gets RuntimeException -> ExecutionException -> the original exception, where the sequential mode throws the original itself: the clause limit (IndexSearcher$TooManyClauses), a fired ExitableDirectoryReader timeout, the IOException every search overload declares, or an Error. Handlers that catch those types do not fire. The search still fails and returns nothing, and the original exception with its message stays in the cause chain. IndexSearcher stopped wrapping the same way in 9.8.0 (#12516) to make sequential and concurrent exceptions the same, and the facet module's other concurrent code already rethrows the cause. Opt-in mode of an @lucene.experimental class; every release from 6.4.0 to 10.6.0 and main.

• Drill down / drill sideways: a DrillDownQuery is a base query plus one filter per drilled dimension. DrillSideways returns its hits and facet counts, and for each drilled dimension the counts computed without that dimension's filter (the "sideways" counts).
• Sequential and concurrent mode: the 3- and 4-argument constructors pass no executor (DrillSideways.java:89-112); the 5-argument one takes an ExecutorService ("Use this constructor to use the concurrent implementation", :114-125). The class javadoc presents the two as alternatives that differ in speed and duplicate work (:57-64).

1. The executor picks the path. Every search overload is declared throws IOException and ends in if (executor != null) searchConcurrently(...) else searchSequentially(...) (DrillSideways.java:352-355, :424-428).
2. Sequential. searchSequentially (:441-481) runs one DrillSidewaysQuery with searcher.search(...) on the caller's thread (:467); a failure propagates as thrown.
3. Concurrent. searchConcurrently (:483-522, no throws clause) wraps each search in a CallableCollector whose call() is searcher.search(query, collectorManager) (:302-310), submits the drill-down task (:506), runs the drill-sideways tasks with invokeAll (:507-508) and reads them with Future.get (:510, :514). Future.get reports a task's exception as an ExecutionException, and the catch wraps that again: catch (ExecutionException e) { throw new RuntimeException(e); } (:519-520).
4. Result. RuntimeException (message: the ExecutionException's toString()) -> ExecutionException -> the original exception. So an IOException thrown inside a search task never reaches the caller as an IOException in this mode (one thrown afterwards on the caller's thread, for example by buildFacetsResult at :390, still can).

Any exception a task throws is wrapped: from rewrite, weight creation, scoring or collection, including what an executor-backed IndexSearcher rethrows from its own slices. Not affected: argument checks and facet building on the caller's thread, a RejectedExecutionException from the executor (propagates unchanged), and interrupts, which take a separate ThreadInterruptedException path (:517-518).

Minimal repro (one empty document, a base query whose createWeight throws IOException("simulated read failure"), one drill-down dimension): sequential java.io.IOException: simulated read failure; with a single-thread executor java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOException: simulated read failure, thrown at DrillSideways.java:520. Finder's test: a base query of 1,026 term clauses (two nested BooleanQuerys of 513; limit 1,024) gives IndexSearcher$TooManyClauses: maxClauseCount is set to 1024 sequentially and RuntimeException: ExecutionException: IndexSearcher$TooManyClauses: ... with a 2-thread pool; a fired ExitableDirectoryReader timeout the same with ExitingReaderException. Verifiers added an IOException from a hit collector (wrapped even when the IndexSearcher has its own executor, which itself throws it as-is), a 1,100-term TermRangeQuery with SCORING_BOOLEAN_REWRITE, and a simulated StackOverflowError.

• Who is hit. Code that runs DrillSideways with an executor and handles failures by type: catch (IndexSearcher.TooManyClauses e) for "query too large", catch (ExitingReaderException e) for "timed out", catch (IOException e), which the signature invites, or treating Error as fatal. In the concurrent mode all of them miss, and the failure takes the generic error path instead.
• Opt-in. Only the 5-argument constructor with a non-null ExecutorService is affected; the 3- and 4-argument constructors are sequential. DrillSideways is @lucene.experimental.
• Effect. For the tested failures, only the outer exception type differs. The search still aborts and returns nothing, every task has finished before the wrapper is thrown (invokeAll and Future.get wait for them, DrillSideways.java:506-516), and the original exception and message stay in the cause chain. (The separate interrupt path does not wait for the drill-down task; it is not part of this bug.) Handlers that catch RuntimeException (as TooManyClauses and ExitingReaderException are) or unwrap the cause chain are not affected.
• Scale. None: one document and any failing task.
• Not tested: identical query semantics. Sequential mode runs one DrillSidewaysQuery, whose visit only calls visitLeaf(this) (DrillSidewaysQuery.java:121-124); concurrent mode runs the DrillDownQuery plus one query per dimension (DrillSideways.java:488-503). Whether a visitor-based check such as the nested clause count can then fail in one mode only was not tested; that would be a separate issue.
• Likelihood: plausible. The triggers are ordinary for user-facing search (a query over 1,024 clauses, a reader timeout under load, an I/O error), but it only bites type-specific handlers, and the recorded searches found no report of it since 2017.

One class, one method (89 lines with the license header). The anonymous Query is the smallest stock-API failure: no clause-limit setup and no timeout wiring. One empty document suffices because the failure happens at weight creation; one drill-down dimension makes the concurrent path run one drill-down and one drill-sideways task. The taxonomy reader and SortedSetDocValuesReaderState are null because the search fails before any facets are built. The stock triggers (the 1,026-clause query, a fired timeout) fail through the same catch.

As a regression test it is the suggested one, unchanged except for one edit that ./gradlew :lucene:facet:check needs: google-java-format wraps the 101-character comment before the second expectThrows onto two lines. With that edit and the fix, the module's checks pass (see Suggested fix).

# main 645d4fe14a20, ./gradlew :lucene:facet:test --tests ...TestDrillSidewaysExecutorExceptionType -Ptests.seed=DEADBEEF -Ptests.asserts=true
# (-Ptests.asserts=false: the same failure); module prefixes of frames and some frames elided
TestDrillSidewaysExecutorExceptionType > testExecutorThrowsSameExceptionAsSequential FAILED
    junit.framework.AssertionFailedError: Unexpected exception type, expected IOException but got java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOException: simulated read failure
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:D357657FD11EB4DD]:0)
        at org.apache.lucene.tests.util.LuceneTestCaseParent.expectThrows(LuceneTestCaseParent.java:1418)
        at org.apache.lucene.facet.TestDrillSidewaysExecutorExceptionType.testExecutorThrowsSameExceptionAsSequential(TestDrillSidewaysExecutorExceptionType.java:80)
        Caused by:
        java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOException: simulated read failure
            at org.apache.lucene.facet.DrillSideways.searchConcurrently(DrillSideways.java:520)
            at org.apache.lucene.facet.DrillSideways.search(DrillSideways.java:353)
            at org.apache.lucene.facet.DrillSideways.search(DrillSideways.java:234)
            at org.apache.lucene.facet.DrillSideways.search(DrillSideways.java:221)
            Caused by:
            java.util.concurrent.ExecutionException: java.io.IOException: simulated read failure
                at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:124)
                at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:193)
                at org.apache.lucene.facet.DrillSideways.searchConcurrently(DrillSideways.java:510)
                Caused by:
                java.io.IOException: simulated read failure
                    at org.apache.lucene.facet.TestDrillSidewaysExecutorExceptionType$1.createWeight(TestDrillSidewaysExecutorExceptionType.java:45)
                    at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:978)
                    ...
:lucene:facet:test (FAILURE): 1 test, 1  failure

# line 80 is the second expectThrows (with the executor); the first, sequential one passed
# 10.6.0 release jars (standalone runner, -ea and -da): the same exception chain

• The javadoc (DrillSideways.java:57-64): "There is both a "standard" and "concurrent" implementation for drill sideways. The concurrent approach is enabled by providing an ExecutorService to the ctor. The concurrent implementation may be a little faster but does duplicate work ... The standard approach may be a little slower but avoids duplicate computations and query processing." Nothing about exceptions. (Added by #142, LUCENE-9944; in every release from 9.0.0 on.)
• The signature. Every search overload declares throws IOException (:188-190, :220, :225-226, :326-328, :406-410); in the concurrent mode an IOException thrown by a search task only ever arrives wrapped in an unchecked RuntimeException.
• #12516 (Luca Cavanna, 9.8.0, CHANGES.txt:3429-3432, "Changes in runtime behavior": "Unwrap and throw execution exceptions cause when performing concurrent search") made TaskExecutor rethrow the cause (TaskExecutor.java:118-134). Its description: "This unifies the exceptions that get thrown between sequential codepath (when no executor is provided) and concurrent codepath (when an executor is provided)." Besides its CHANGES.txt entry it changed only core files; it touched no facet file and never mentions DrillSideways.
• The module's convention. ConcurrentSortedSetDocValuesFacetCounts.java:310-313 and :342-345, and DynamicRangeUtil.java:92-93, rethrow the cause with IOUtils.rethrowAlways.
• Tests. TestParallelDrillSideways extends TestDrillSideways and only swaps in the executor constructors, so the whole suite runs in both modes; no test makes a DrillSideways.search call fail, and no facet test mentions ExecutionException, so neither shape is pinned.
• Intent at introduction. In LUCENE-7588 (#8639) mikemccand asked to fold the parallel version into DrillSideways: "It's really an implementation/execution detail I think? Similar to how IndexSearcher takes an optional executor." The thread never discusses how task failures reach the caller.

Strongest case (refute review, the cross-model review, the intended-behaviour check, adjudication): DrillSideways is @lucene.experimental; no javadoc promises the same exception types in both modes, and throws IOException does not forbid unchecked wrappers; the wrapping is as old as the concurrent mode (2017) and matched IndexSearcher until 9.7.0, where TestLRUQueryCache still has the branch "expected: wrapped when executor is in use" (TestLRUQueryCache.java:1500-1504); and upstream filed #12516 under "Changes in runtime behavior", not "Bug Fixes".

Why it fails: the wrapping was never documented or discussed as a choice; it matched the IndexSearcher of its time, whose maintainers later removed the wrapping as the "root cause" (javanna, #12516 review) of mismatched exceptions; the module's other concurrent code unwraps; and the declared IOException, when a search task throws it, cannot arrive as one. The verifiers rated the case moderate as history and weak as a defence. Verdict: not intended; a low-priority consistency fix following #12516.

No-noise check. The call fails loudly either way and the cause is preserved. It is still claimed, as low priority, because the inputs are ordinary (clause limit, timeout, I/O error), the effect is visible to users (type-based handlers take the wrong branch), and Lucene treated the identical pattern in IndexSearcher as worth changing. The hunt's noise audit rated it "claim-low".

Option A was run through gradle on a scratch worktree of 645d4fe14a20 (seed DEADBEEF); B and C were not run.

+6/-2 in DrillSideways.java: import org.apache.lucene.util.IOUtils, declare throws IOException on searchConcurrently (both callers already declare it), and rethrow the ExecutionException's cause, null-guarded as in ConcurrentSortedSetDocValuesFacetCounts.java:310-313. IOUtils.rethrowAlways throws an IOException, RuntimeException or Error as itself and wraps anything else in a RuntimeException (IOUtils.java:495-513); a task here can only throw what IndexSearcher.search throws. This is the shape #12516 gave TaskExecutor, where jpountz suggested IOUtils.rethrowAlways in review.

      final List<? extends CollectorManager<K, R>> drillSidewaysCollectorManagers)
      throws IOException {
  ...
    } catch (ExecutionException e) {
      // Rethrow the task's own exception, as the sequential path and IndexSearcher do
      Throwable cause = e.getCause();
      throw IOUtils.rethrowAlways(cause != null ? cause : e);
    }

Trade-offs

• A behaviour change for callers that today unwrap RuntimeException -> ExecutionException by hand: they now get the exception itself. #12516 shipped the same change under "Changes in runtime behavior", where a CHANGES entry belongs.
• When several tasks fail, only one exception is thrown, as today (the drill-down task's, else the first failing drill-sideways task's); the others are not added as suppressed.
• Interrupt handling is unchanged.

What the checks validated

• The repro passes with assertions on and off (1 test each).
• TestDrillSideways (14 tests), TestParallelDrillSideways (14) and the repro: 29 tests, 0 failures.
• The whole lucene/facet test suite: 40 suites, 307 tests, 8 skipped, 0 failures.
• ./gradlew :lucene:facet:check -x test (google-java-format, ecj lint, forbidden APIs) passes with the fix and the test, once its 101-character comment is wrapped onto two lines; unwrapped, only google-java-format objects.
• The wrapped (PR-ready) copy passes with the fix with assertions off; it was not run with assertions on, and differs from the repro only in that comment.
• Earlier, the independent reproduction's equivalent fix (+3/-2) made the independent reproduction's four-trigger test, the finder's test and the history probe pass on a9779ef3c193.

new TaskExecutor(executor).invokeAll(...) unwraps the cause and adds secondary failures as suppressed (TaskExecutor.java:118-134).

Trade-offs

TaskExecutor also runs tasks on the calling thread (TaskExecutor.java:103-114), which changes how DrillSideways uses the caller's executor: a larger change than the bug needs.

A javadoc note on the executor constructor.

Trade-offs

Settles the contract without a behaviour change, but keeps the declared IOException unreachable and the mismatch with the sequential mode, IndexSearcher and the rest of the facet module.

Method: semi-formal (the hunt's tag; an enumerated differential sweep, borderline with ordinary differential testing).

• A hunt unit (executor pluggable-behaviour sweep), 2026-10-26, on main a9779ef3c193: 13 API surfaces x 9 executor behaviours (caller-runs, single thread, rejecting at the 1st/2nd/3rd execute, shut down, LIFO pool, pool thread interrupted before task 1 or 2) = 117 cells, each compared with the executor-less call: same result, same exception class and message, returns within 8 s, executor terminates.
• The row "DrillSideways.search(failing-base)", a base query that throws IOException on one segment, differed in all 9 executor cells: IOException sequentially, RuntimeException(ExecutionException(IOException)) (or the executor's own RejectedExecutionException for the rejecting executors) with an executor. The successful DrillSideways path matched in every non-rejecting cell.
• The finder reduced it to two stock triggers, the clause limit and an ExitableDirectoryReader timeout, in a public-API JUnit (2 of 2 fail with assertions on and off); the minimal repro step later reduced that to the one-method test above.