IndexWriter.deleteDocuments(Term...) and deleteDocuments(Query...) keep the caller's array, so reusing it after the call deletes the wrong documents

IndexWriter.deleteDocuments(Term...) and deleteDocuments(Query...) hand the caller's array itself to the writer's delete queue. During the call the queue copies the elements into the buffer that serves segments already flushed, so those documents are deleted by what the array held at the call. Each per-thread writer applies the delete to the documents it still buffers in RAM only later, at its next addDocument or at flush, and reads the array as it is then.

So a caller that refills or reuses its array after the call deletes buffered documents by the new contents. With the default IndexWriterConfig, a loop that refills one Term[2] with {d0, d1}, {d2, d3}, {d4, d5} commits [d0, d1, d2, d3, keep] instead of [keep]; the same loop with a new array per call, or with the documents flushed first, commits [keep]. Reassigning queries[0] after the call deletes a document no call named. Clearing a Query[] after the call makes the next commit a tragic writer failure that loses all uncommitted documents. Every release from 4.0.0-ALPHA (2012) through 10.5.2 and main is affected; a 2-line copy in IndexWriter fixes it.

• RAM buffer, per-thread writer: IndexWriter collects new documents in memory, in one or more per-thread writers, until a flush writes them as a new segment. A flush happens at commit(), when an NRT reader is opened or refreshed, when the RAM buffer is full (16 MB by default) or on flush().
• Delete queue: deletes are appended to one shared queue. A global slice of it serves the segments that are already flushed; each per-thread writer has its own delete slice for the documents it buffers.
• Node: one queue entry per deleteDocuments call, so all of a call's deletes become visible together.

1. deleteDocuments(Term... terms) calls docWriter.deleteTerms(terms) (IndexWriter.java:1867); the Query... overload calls docWriter.deleteQueries(queries) (:1894). DocumentsWriter passes the array on (DocumentsWriter.java:141-147), and DocumentsWriterDeleteQueue.addDelete appends new TermArrayNode(terms) / new QueryArrayNode(queries) (DocumentsWriterDeleteQueue.java:123-133). The node stores the array itself, not a copy (:462-465, :475-478).
2. Flushed segments, call-time contents. addDelete then calls tryApplyGlobalSlice() (:125, :131). When its tryLock succeeds (always with one thread) it applies the node to the global buffer during the call (:198-215), copying each element (:467-472, :480-485). That buffer is applied to segments already flushed.
3. Buffered documents, later contents. A per-thread writer applies its slice only when it finishes its next document (DocumentsWriterPerThread.java:361-389, apply at :377 and :383) or when it flushes (prepareFlush, :427-436, apply at :434). The node's apply loops over the stored array as it is at that moment (for (Term term : item), DocumentsWriterDeleteQueue.java:482).
4. The batch loop. Three calls with one array add three nodes that point at the same array. At commit the per-thread writer applies all three; each reads {d4, d5}. So the {d4, d5} terms are added three times (BufferedUpdates.addTerm keeps one entry per term, so d4 and d5 are deleted once) and d0 to d3 are never deleted.
5. Clearing the array. A null slot reaches BufferedUpdates.addTerm at the next addDocument (which throws a NullPointerException, and the delete is dropped), or, for queries, IndexSearcher.rewrite while the commit applies the segment's deletes; an exception there is a tragic event (IndexWriter.java:2933-2942) that closes the writer.

For one indexing thread the window closes at that writer's next indexed document (any add or update call, such as addDocument or updateDocument) or at the next flush (such as commit(), an NRT reopen or flush()); another delete call does not close it. With several threads each per-thread writer applies the node at its own next document, and if tryApplyGlobalSlice misses its tryLock, flushed segments read the array late too; both come from code reading only, every run was single-threaded.

• Who is hit. Code that passes an explicit Term[] or Query[] to deleteDocuments and then changes it before that per-thread writer indexes another document or flushes: refilling one buffer per batch, clearing it with Arrays.fill(buf, null), or reusing it for something else. Literal varargs (deleteDocuments(t), deleteDocuments(t1, t2)) get a fresh array from the compiler, and code that builds a new array per call is safe.
• Default configuration, mainstream API. The repro uses the default IndexWriterConfig, codec and merge policy and one thread; nothing is opted into.
• Scale. Every document added since the last flush can be affected; the repros use 2 to 7 documents because size does not matter.
• Silent and durable. The call returns its sequence number, there is no exception, and the commit persists both errors: documents the caller deleted stay searchable (also through NRT readers), and documents nobody named are gone. No structural corruption was observed (readers open the committed index normally; CheckIndex was not run).
• Clearing is worse. After deleteDocuments(Query[]), Arrays.fill(buf, null) makes the next commit() throw a NullPointerException; the writer closes on the tragic event and every uncommitted document is lost (committed live set []). After deleteDocuments(Term[]), the next unrelated addDocument throws a NullPointerException and the delete is dropped (the writer stays open).
• How likely. Rare. Code search found no Lucene module and no Elasticsearch, OpenSearch or Solr main-code call site that reuses an array (they pass one argument or build a new array per call), and nobody has reported it since 4.0. Code search can miss call sites.

# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
    => java.lang.AssertionError: expected:<[keep]> but was:<[delete-me]>
       org.junit.Assert.fail(Assert.java:89)
       org.junit.Assert.failNotEquals(Assert.java:835)
       org.junit.Assert.assertEquals(Assert.java:120)
       org.junit.Assert.assertEquals(Assert.java:146)
       org.apache.lucene.index.TestDeleteDocumentsReusedQueryArray.testReusedQueryArray(TestDeleteDocumentsReusedQueryArray.java:57)

• The contract speaks of the deletes given. "Deletes the document(s) containing any of the terms. All given deletes are applied and flushed atomically at the same time." and "@param terms array of terms to identify the documents to be deleted" (IndexWriter.java:1856-1860; the same for queries at :1875-1879). Nothing says the array is kept or must not be changed.
• Lucene documents retention in the other places checked where it keeps caller data. MultiPhraseQuery.Builder.add(Term[]): "The array is not copied or mutated, the caller should consider it immutable subsequent to calling this method" (MultiPhraseQuery.java:111-112, :122-123). Field/StoredField: "the provided byte[] is not copied so be sure not to change it until you're done with this field" (Field.java:156, :172; StoredField.java:68). The Term objects passed in are already protected: "The provided BytesRef is copied when it is non null." (Term.java:50).
• The result depends on flush timing. In one call, flushed documents see the call-time contents and buffered documents the later contents. The class javadoc presents flushing as buffering ("These changes are buffered in memory and periodically flushed", IndexWriter.java:123) and says a change's sequence number "expresses the effective order in which each change was applied" (:117-118); here what the change deletes is decided after its number is assigned.
• History. 3.x applied the elements inside the call (Term[] deletes since 2.1.0, Query[] since 2.4.0). Retention came with the new (then lock-free) delete queue of LUCENE-2956 (#4030, 2011), whose review asked that a Term[]/Query[] delete be atomic, hence one node per call; nobody discussed copying or reusing the array.
• Tests. No upstream test reuses an array at the public API. The only identity assertion is the package-private queue test TestDocumentsWriterDeleteQueue.testUpdateDeleteSlices ("create an array here since we compare identity below against tailItem", TestDocumentsWriterDeleteQueue.java:51-56); it passes with the boundary copy.
• Precedent. LUCENE-6821 (#7879, 2015) made Term copy its BytesRef because the aliasing had "bitten a few people"; #12158 (2023) cloned a caller's BytesRef[] in KeywordField#newSetQuery.

The strongest case for "works as designed" (the intended-behaviour checker and an intended-lens review by OpenAI GPT-5.6): IndexWriter buffers its changes, Java does not require defensive copies, and the delete queue keeps its payload on purpose (an internal test asserts identity), so a caller must not modify an array after passing it in.

Why it fails: the public methods say nothing, while Lucene documents retention in the other places checked where it keeps caller data (MultiPhraseQuery.Builder, Field, StoredField); within one call the outcome depends on whether a document was flushed; the identity test covers the internal queue only; upstream removed the same kind of trap by copying; and a routine cleanup (Arrays.fill(buf, null)) turns into a tragic writer failure. The intended-behaviour checker remains only moderately sure about the remedy: upstream may prefer a javadoc note to a copy.

One fix was built and validated; the alternatives come from the finder, the reviewers and the checkers.

Clone the varargs array before it is handed to the delete queue, in both overloads (IndexWriter.java:1867, :1894). The Term and Query objects are not copied; Term already copies its bytes.

   public long deleteDocuments(Term... terms) throws IOException {
     ensureOpen();
     try {
-      return maybeProcessEvents(docWriter.deleteTerms(terms));
+      // copy: the deletes are resolved against buffered documents after this call returns
+      return maybeProcessEvents(docWriter.deleteTerms(terms.clone()));
@@ deleteDocuments(Query... queries)
-      return maybeProcessEvents(docWriter.deleteQueries(queries));
+      // copy: the deletes are resolved against buffered documents after this call returns
+      return maybeProcessEvents(docWriter.deleteQueries(queries.clone()));

Trade-offs

One shallow copy of the varargs array per call (one reference per term or query), not per document. It also removes the cleared-array failures. In the Query... overload the copy is taken after the MatchAllDocsQuery check, so another thread could still change the array between the two; a variant, not run, clones once at the top and uses the copy for both the check and the delete. The copy covers the array slots only: a caller that later mutates a custom Query object it passed in is still exposed.

What the checks validated

• In a scratch worktree of a9779ef3c193, through gradle: stock fails the test 3 of 3 with assertions on and off; with the copy it passes 3 of 3 with assertions on and off.
• 20 closest suites pass with assertions on and off (347 tests, 18 skipped, 0 failed each): TestIndexWriterDelete, TestDocumentsWriterDeleteQueue, TestIndexWriter, TestIndexWriterReader, TestIndexWriterExceptions, TestIndexWriterExceptions2, TestStressDeletes, TestStressNRT, TestRollingUpdates, TestTryDelete, TestIndexingSequenceNumbers, the doc-values update suites and others. The whole core index package: 215 suites, 1,398 tests, 0 failed.
• With the copy shadowed, the plain program, the finder's and the independent reproducer's tests and the security probe all pass.
• checkGoogleJavaFormat passes with the fix and the test in place.
• Not run: the full test suite and any benchmark.

Store terms.clone() / queries.clone() in the queue node (DocumentsWriterDeleteQueue.java:123-133).

Trade-offs

Also removes the bug (shown at the introducing commit), but it fails the internal identity assertion of TestDocumentsWriterDeleteQueue.testUpdateDeleteSlices (1 failure; stock passes), so that test would change too. The only production entry path is IndexWriter, through DocumentsWriter (package tests call addDelete directly), so it gains nothing over the boundary copy today.

What the checks validated

Executed: the queue-level copy fails testUpdateDeleteSlices and passes TestIndexWriterDelete; at the introducing commit it turns the failing scenario into a pass.

Add "The array is not copied; do not modify it after this call" to both javadocs, in the style of MultiPhraseQuery.Builder.add(Term[]).

Trade-offs

Zero cost, but the trap stays: the flush-timing-dependent result, the lost and unrequested deletes, and the tragic commit after a cleared Query[]. Existing callers that reuse arrays stay broken.

What Lucene 3.x did.

Trade-offs

Gives up the shared-queue, one-node-per-call design of the delete queue for no gain over the boundary copy.

Method: an argument-ownership table checked by mutate-after-return probes (the study's "semi-formal" tier). A hunt unit (IndexWriter update-value ownership), leg 2.

• From code reading, the AI tool built a table of every IndexWriter entry point that takes a caller-owned mutable object (arrays, BytesRefs, fields) and classified each argument as copied, consumed inside the call, or kept.
• Each row was witnessed by an executed probe on the real main jars: change the argument after the call returned, then compare a document buffered in RAM with one in a flushed segment.
• For deleteDocuments(Term...) the same call deleted the flushed document and not the buffered one. The AI tool minimised that to the 2-test JUnit repro, and the fix and the checks followed.
• No model checker, solver or prover was involved; the table guided where to probe.