In NRT replication a replica copies each new point's files into temp files. If a newer point arrives mid-copy, the new copy job keeps the temp files the old job already finished, matched by file name only. After a failover, the new primary can write a different file under the same name.
.liv files, the temp file B keeps, and what B shows at version 9 against what P2 has. Select the still to open it at full size. Generated with AI, checked against the code.- You run NRT replication: primary P1, replicas A and B, segments
_0= {a0, b0} and_1= {a1, b1}. P1 deletes a0 and a1 and publishes a point with two new delete files,_0_1.livand_1_1.liv. B's copy job finishes P1's_1_1.liv; then P1 goes down (the test closes it without waiting for copies). - A is promoted to primary P2. P2 deletes b0 and b1 and publishes version 9 with its own
_0_1.livand_1_1.liv: the same names, one delete each, other bits. - You expect B, once it is at version 9, to show P2's documents: a0 and a1.
- Instead B hears of P2 before its old copy job has failed, and
CopyJob.transferAndCancelkeeps the finished temp file because the name matches; B copies only P2's_0_1.liv. At version 9, B shows a0 and b1: b1, deleted on P2, is visible, a1 is hidden, and the hit count is still 2. - B commits that state and
CheckIndexreports no problems. By reading the code, the stale file stays until that segment gets a newer delete file or is merged away; a restart alone does not remove it. Reproduced on 12 releases from 6.0.0 to 10.5.2 and on main.
Only applications on the opt-in, experimental NRT replication classes that fail over. When the new primary's first point reaches a replica, its copy job from the old primary must still be alive and have finished a file the new primary reuses by name. Lucene's own stress test never does that; any index size, default codec.
Mechanism, terms and numbers869 words
In lucene-replicator's NRT segment replication a replica copies a new point's files into temp files. When a newer point arrives while a copy runs, ReplicaNode.newNRTPoint calls CopyJob.transferAndCancel, which keeps every temp file the old job has finished whose name the new job also needs. It compares names only, never the files' FileMetaData, and it does so even when the newer point comes from a new primary. After a failover the new primary can write a different file under the same name, such as the live-docs file _1_1.liv of a segment both primaries share. The replica then reports the new primary's version but searches a mix of both primaries' deletes ([a0, b1] instead of [a0, a1] in the repro: a deleted document visible, a live one hidden, the same hit count), commits it, and CheckIndex reports the index clean. It needs no split brain: the old primary can be gone before the promotion. It was reproduced on 12 releases from 6.0.0 to 10.5.2 and on main (the releases in between have the same code), and it affects only applications that use the opt-in, experimental NRT replication classes and fail over while a replica's copy job from the old primary is still alive.
- Primary and replicas:
PrimaryNodeindexes; eachReplicaNodecopies the primary's new files and opens them. A failover promotes a replica to primary with a higher primaryGen. - NRT point: a searchable state with a version number. For each point, a replica runs a copy job (
CopyJob) that copies the files it lacks into temp files and renames them when the job finishes. .livfile: a segment's live-docs (deletes) file, named by segment and delete generation:_1_1.livis the first delete generation of segment_1. Two primaries that each delete one document of the shared segment_1both write_1_1.liv, with different bits.
ReplicaNode.newNRTPointfirst callsmaybeNewPrimary(ReplicaNode.java:545). On a primary change it discards only pre-copied merge files; the comment above says why: "it's still a bit risky to rely solely on checksum/file length to catch the difference, so we defensively discard here and re-copy" (:536-544).- It builds the new job, whose
toCopylists the new point's files that the replica lacks (getFilesToCopy,ReplicaNode.java:834-847); a temp file is not a local file, so both.livfiles are listed. - With a job in flight it calls
job.transferAndCancel(curNRTCopy)(:613-616), whatever the jobs' primaryGens. CopyJob._transferAndCancel, unless the old job already failed (CopyJob.java:119-123), keeps every temp file the old job finished whose name the new job wants:prevJob.copiedFiles.get(fileName) != nullmoves it over and drops the name fromtoCopy(:133-156). Which of the two files the old job finished, and so which is carried, follows the job's iteration order; here_1_1.liv, in the independent reproducer's run_0_1.liv. The old and newFileMetaDataare never compared.- The replica renames the old primary's temp file to
_1_1.livand opens the new primary's version with it. - Nothing notices.
CopyOneFilechecks checksums only while it copies bytes (CopyOneFile.java:99-126). The carried file is genuine (its footer matches its bytes), its header has the shared segment's id and the same generation, and both primaries deleted one document there, so the reader's header, footer and delete-count checks pass (Lucene90LiveDocsFormat.java:86-92,:98,:123-126). - It stays (by reading, not executed):
finishNRTCopycaches the new point's metadata as the replica's own (ReplicaNode.java:475) andreadLocalFileMetaDataanswers from that cache (Node.java:196-205), so later points do not re-copy the stale file until the segment gets a new delete generation or is merged away. A restart alone does not repair it: the commit records primaryGen 2 (ReplicaNode.java:367-373), so a restart with the same primary reopens that commit without a sync (:189,:301-305,:317-318); the restart clears the cache, so the next point with a higher version compares the file on disk and copies it again.
| replica B at the new primary P2's version 9 | documents |
|---|---|
| expected: P2's documents | [a0, a1] |
searched on main (-ea, -da) and 10.5.2 | [a0, b1] |
| committed, reopened | [a0, b1], CheckIndex clean |
| control: B's old job finished nothing before the failover | [a0, a1] |
| with the recommended fix | [a0, a1] |
An independent reproducer also got the mirror case, [a1, b0] with _0_1.liv carried over, and showed that the replica's .liv footer checksum equals the old primary's, not the new one's.
Not silent, by reading: a reused new-segment name (.si, .cfs) fails the segment-id check at reader open; a .liv with another delete count throws CorruptIndexException; the in-progress branch (CopyJob.java:157-193) resumes with the old metadata, so a content change fails the checksum ("checksum mismatch after file copy", CopyOneFile.java:99-111) and the copy job is cancelled; nothing retries that point (the module's reference job runner, test code, cancels and moves on, Jobs.java:76-96; finishNRTCopy returns for a failed job, ReplicaNode.java:425-426), so the replica stays at its old version until a later point. Doc-values update files of a shared segment, including a soft-deletes field, take the same name-only branch (not executed).
Exposure and likelihood479 words
- Who: applications that use
lucene-replicator's NRT segment replication (PrimaryNode,ReplicaNodeand their ownCopyJobsubclass, all@lucene.experimental) and fail over. One open-source downstream user was found, nrtsearch; its replica node and copy jobs inherit this code (read only, not run); whether its failover sequencing and copy timing can open the window was not examined. Elasticsearch, OpenSearch and Solr use their own replication. - The window: when the replica gets the new primary's first NRT point, its copy job from the old primary has not failed yet and has finished a file that the new primary reuses by name with other content. Orderings that open it: the old primary is gone before the promotion and the job has not noticed yet (this doc's test, which closes P1 without waiting for copies); a deposed primary that keeps publishing after the promotion (partition or long pause; executed by the finder); a half-open connection; a copy job idle between files; a controller that promotes without draining copies.
- Narrow with the reference copy job: the module's socket-based test
SimpleCopyJobholds its monitor while it reads (SimpleCopyJob.java:184), so against a dead primary the carry-over only wins a race between the failed read and the cancel (verifiers' reading, not run). Applications supply their ownCopyJob, so their window depends on it. Every no-split-brain reproduction keeps the old job alive by construction (a no-oplaunch(), one file copied by hand, then the new point before the job reads again): legal for aCopyJobsubclass, but no networked job has reproduced it. - Why Lucene's tests miss it:
TestStressNRTReplicationcrashes the primary's process (its socket copy jobs then normally fail on their next read) and waits for in-flight copies before it promotes (SimpleReplicaNode.java:222-234); a TLA+ model finds no violation under that discipline (220 states). Its hit-count oracle would not notice either (2 = 2). - Silent condition: the reused file must pass the reader's checks; in every executed run it was a
.livof a shared segment with the same delete count. A different count fails loudly. - Damage: the replica reports the new primary's version but serves a mix of both primaries' deletes, commits it, and
CheckIndexis clean. By reading: it lasts until the segment gets a new delete generation or is merged away, and if that replica is promoted it becomes the authoritative index. - Scale: none; two segments of two documents suffice. Default codec, default
IndexWriterConfigapart fromNoMergePolicy(set only to keep the two segments), ordinary hard deletes; for silence the same-named file must also pass the reader's checks (above). - How likely: rare (a judgement): it needs that failover timing plus a same-named generation file with the same delete count, and no matching public report was found in the documented searches. Not contrived: each step is an ordinary failover event.
The test shows that after a failover (P1 crashes, replica A becomes P2), replica B keeps a .liv file it had already copied from P1 because P2's file has the same name, so B at P2's NRT version serves a mix of both primaries' deletes.
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.lucene.replicator.nrt;
import java.io.IOException;
import java.nio.file.Path;
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
import java.util.TreeSet;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.document.StringField;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.index.NoMergePolicy;
import org.apache.lucene.index.SegmentCommitInfo;
import org.apache.lucene.index.StoredFields;
import org.apache.lucene.index.Term;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.MatchAllDocsQuery;
import org.apache.lucene.search.ScoreDoc;
import org.apache.lucene.store.FSDirectory;
import org.apache.lucene.store.IOContext;
import org.apache.lucene.store.IndexInput;
import org.apache.lucene.tests.util.LuceneTestCase;
import org.apache.lucene.tests.util.LuceneTestCase.SuppressFileSystems;
import org.apache.lucene.tests.util.TestUtil;
@SuppressFileSystems("*")
public class TestNRTCopyAcrossPrimaryChange extends LuceneTestCase {
public void testReplicaMatchesNewPrimaryAfterFailover() throws Exception {
Path aPath = createTempDir();
IndexWriter w1 = new IndexWriter(FSDirectory.open(createTempDir()), iwc());
Primary p1 = new Primary(w1, 0, 1, -1);
w1.addDocument(doc("a0"));
w1.addDocument(doc("b0"));
w1.flush();
w1.addDocument(doc("a1"));
w1.addDocument(doc("b1"));
p1.flushAndRefresh();
Replica ra = new Replica(1, aPath, p1);
Replica rb = new Replica(2, createTempDir(), p1);
copyAll(ra.newNRTPoint(1, p1.getCopyStateVersion()));
copyAll(rb.newNRTPoint(1, p1.getCopyStateVersion()));
w1.deleteDocuments(new Term("id", "a0"), new Term("id", "a1"));
p1.flushAndRefresh();
long v2 = p1.getCopyStateVersion();
((Job) rb.newNRTPoint(1, v2)).copyNextFile();
p1.close();
ra.commit();
ra.close();
IndexWriter w2 =
new IndexWriter(
FSDirectory.open(aPath), iwc().setOpenMode(IndexWriterConfig.OpenMode.APPEND));
Primary p2 = new Primary(w2, 1, 2, v2 + 1);
w2.deleteDocuments(new Term("id", "b0"), new Term("id", "b1"));
p2.flushAndRefresh();
rb.primary = p2;
copyAll(rb.newNRTPoint(2, p2.getCopyStateVersion()));
IndexSearcher searcher = rb.getSearcherManager().acquire();
try {
Set<String> ids = new TreeSet<>();
StoredFields storedFields = searcher.getIndexReader().storedFields();
for (ScoreDoc hit : searcher.search(new MatchAllDocsQuery(), 10).scoreDocs) {
ids.add(storedFields.document(hit.doc).get("id"));
}
// expected: B, now at P2's version, shows P2's live docs; actual: one .liv is still P1's
assertEquals(new TreeSet<>(Set.of("a0", "a1")), ids);
} finally {
rb.getSearcherManager().release(searcher);
rb.close();
p2.close();
}
}
private static IndexWriterConfig iwc() {
return new IndexWriterConfig()
.setCodec(TestUtil.getDefaultCodec())
.setMergePolicy(NoMergePolicy.INSTANCE);
}
private static Document doc(String id) {
Document doc = new Document();
doc.add(new StringField("id", id, Field.Store.YES));
return doc;
}
private static void copyAll(CopyJob job) throws IOException {
while (((Job) job).copyNextFile()) {}
job.onceDone.run(job);
}
private static final class Primary extends PrimaryNode {
Primary(IndexWriter writer, int id, long primaryGen, long forcePrimaryVersion)
throws IOException {
super(writer, id, primaryGen, forcePrimaryVersion, null, null);
setRemoteCloseTimeoutMs(0);
}
@Override
protected void preCopyMergedSegmentFiles(
SegmentCommitInfo info, Map<String, FileMetaData> files) {}
}
private static final class Replica extends ReplicaNode {
Primary primary;
Replica(int id, Path path, Primary primary) throws IOException {
super(id, FSDirectory.open(path), null, null);
this.primary = primary;
start(1);
}
@Override
protected CopyJob newCopyJob(
String reason,
Map<String, FileMetaData> files,
Map<String, FileMetaData> prevFiles,
boolean highPriority,
CopyJob.OnceDone onceDone)
throws IOException {
CopyState copyState = primary.getCopyState();
return new Job(reason, copyState, this, primary, highPriority, onceDone);
}
@Override
protected void launch(CopyJob job) {}
@Override
protected void sendNewReplica() {}
}
private static final class Job extends CopyJob {
private final Primary src;
private final CopyState copyState;
Job(
String reason,
CopyState copyState,
ReplicaNode dest,
Primary src,
boolean highPriority,
OnceDone onceDone)
throws IOException {
super(reason, copyState.files(), dest, highPriority, onceDone);
this.src = src;
this.copyState = copyState;
}
boolean copyNextFile() throws IOException {
if (toCopy.isEmpty()) {
return false;
}
Map.Entry<String, FileMetaData> next = toCopy.removeFirst();
try (IndexInput in = src.getDirectory().openInput(next.getKey(), IOContext.DEFAULT)) {
CopyOneFile file =
new CopyOneFile(in, dest, next.getKey(), next.getValue(), new byte[8192]);
while (file.visit() == false) {}
copiedFiles.put(file.name, file.tmpName);
}
return true;
}
@Override
public void finish() throws IOException {
for (Map.Entry<String, String> ent : copiedFiles.entrySet()) {
dest.dir.rename(ent.getValue(), ent.getKey());
}
copiedFiles.clear();
}
@Override
public boolean getFailed() {
return exc != null;
}
@Override
public Set<String> getFileNamesToCopy() {
Set<String> names = new HashSet<>();
for (Map.Entry<String, FileMetaData> ent : toCopy) {
names.add(ent.getKey());
}
return names;
}
@Override
public Set<String> getFileNames() {
return files.keySet();
}
@Override
public CopyState getCopyState() {
return copyState;
}
@Override
public long getTotalBytesCopied() {
return 0;
}
@Override
public int compareTo(CopyJob other) {
return Long.compare(ord, other.ord);
}
@Override
public void start() {}
@Override
protected CopyOneFile newCopyOneFile(CopyOneFile current) {
throw new UnsupportedOperationException();
}
@Override
public void runBlocking() {
throw new UnsupportedOperationException();
}
@Override
public boolean conflicts(CopyJob other) {
throw new UnsupportedOperationException();
}
}
}
# in a copy of this finding's repro folder (repro/LFV-241/, which holds src/)
# Lucene checkout at the tested commit, built with: ./gradlew --no-daemon assemble (JDK 25)
LUCENE=/path/to/lucene; V=12.0.0-SNAPSHOT
CP=$LUCENE/lucene/core/build/libs/lucene-core-$V.jar
CP=$CP:$LUCENE/lucene/codecs/build/libs/lucene-codecs-$V.jar
CP=$CP:$LUCENE/lucene/test-framework/build/libs/lucene-test-framework-$V.jar
CP=$CP:$LUCENE/lucene/facet/build/libs/lucene-facet-$V.jar
CP=$CP:$LUCENE/lucene/replicator/build/libs/lucene-replicator-$V.jar
CP=$CP:randomizedtesting-runner-2.9.2.jar:junit-4.13.2.jar:hamcrest-3.0.jar # test-framework dependencies (Maven Central)
CP=$CP:junit-platform-console-standalone-6.1.3.jar # JUnit Platform console launcher (Maven Central)
javac -d classes -cp "$CP" $(find src -name '*.java')
# run 1: assertions on, seed DEADBEEF
java -ea -esa --add-modules jdk.incubator.vector --enable-native-access=ALL-UNNAMED -Dtests.seed=DEADBEEF \
-cp "classes:$CP" org.junit.platform.console.ConsoleLauncher \
execute --select-class org.apache.lucene.replicator.nrt.TestNRTCopyAcrossPrimaryChange
# the module(s) facet, replicator also need their third-party runtime jars
- Expected vs actual
- Expected B to show P2's live docs
[a0, a1]; actual[a0, b1](expected:<[a0, a1]> but was:<[a0, b1]>), with-eaand-da; it passes with the name+metadata carry-over fix.
More files, environment, full output36 words
| lucene | Lucene main e1faeba08bde (2026-11-02) |
|---|---|
| jdk | 25.0.3+9-LTS |
# Lucene main 645d4fe14a20, assertions on (-ea), seed DEADBEEF: 1 test(s) run, 1 failed
=> java.lang.AssertionError: expected:<[a0, a1]> but was:<[a0, b1]>
org.junit.Assert.fail(Assert.java:89)
org.junit.Assert.failNotEquals(Assert.java:835)
org.junit.Assert.assertEquals(Assert.java:120)
org.junit.Assert.assertEquals(Assert.java:146)
org.apache.lucene.replicator.nrt.TestNRTCopyAcrossPrimaryChange.testReplicaMatchesNewPrimaryAfterFailover(TestNRTCopyAcrossPrimaryChange.java:87)- On a primary change,
newNRTPointdiscards pre-copied data because "it's still a bit risky to rely solely on checksum/file length" (ReplicaNode.java:536-544); the carry-over trusts the bare name. - The module's identity rule is name plus header plus footer (
FileMetaDatajavadoc,fileIsIdentical: "Segment name was reused! ... otherwise devastating"). - No drain or fencing precondition is documented, and it reproduces without split brain.
The full argument307 words
- Contracts. The
newNRTPointcomment (ReplicaNode.java:536-544): on a primary change pre-copied data is discarded because "it's still a bit risky to rely solely on checksum/file length to catch the difference, so we defensively discard here and re-copy".fileIsIdentical(:849-873): a local file stands in for the primary's only if "its header (includes the segment id), length, footer (including checksum)" match, with "Segment name was reused! This is rare but possible and otherwise devastating".FileMetaDatajavadoc (FileMetaData.java:20-23): "Header and footer of the file must be identical between primary and replica to consider the files equal".newCopyJobjavadoc (ReplicaNode.java:502-507): a new job may salvage files "shared with the new copy job".PrimaryNode(PrimaryNode.java:116-119) forces the version forward after a crash, "else versions can be illegally reused but show different results"; here one version shows different results on two nodes. - Design. LUCENE-5438 (#6501), by @mikemccand: a file is "the same" across primary and replica if "its file name is the same, its full byte[] index header and footer are identical, and it's in a "known to be fsync'd" state"; after a promotion "file names can be re-used (Lucene is no longer write-once)".
- Tests. No upstream test targets
transferAndCancelacross a primaryGen change; the stress test's discipline avoids it (TLA+ model, 220 states).TestStressNRTReplicationasserts that "all nodes show the same hit count for the same version, i.e. they really are replicas of one another" (:1118-1119), and itspromoteReplicacomment notes that "file name reuse can cause a replication to fail" (:393-397). - History. The carry-over and the unconditional
transferAndCancelcame with the package (1ae7291429b, 2016).9afa560054a(same branch, "fix one stress test failure") added the defensive discard on a primary change, but only for pre-copied merge files. No "won't fix" or "by design" statement was found.
The devil's-advocate check251 words
The strongest case for "intended or misuse" (built by a devil's-advocate verifier; reviews by an independent reviewer from another model family, OpenAI GPT-5.6, rated it moderate): the module is @lucene.experimental; replicas do not fence a deposed primary (maybeNewPrimary only asserts a higher primaryGen); the reference controller kills the old primary and drains copies before promoting; and the transferAndCancel javadoc literally says it "Transfers whatever tmp files were already copied in this previous job" (CopyJob.java:97-100).
Why it fails: no javadoc, package doc, CHANGES or MIGRATE entry states a drain or fencing precondition, and the drain lives in test code with another stated purpose ("to be sure we really pick the most current replica"); newNRTPoint deliberately accepts a higher primaryGen while a job is in flight and right there states a policy of not trusting old-primary files; the module's own identity rule is name plus header plus footer; and split brain is not needed, since the old primary can be gone before the promotion. What the argument does establish: the reference controller in effect drains (it waits while a replica is copying, SimpleReplicaNode.java:222-238); a default PrimaryNode.close() waits for outstanding copies without limit (PrimaryNode.java:207-216, :339-359), and the test sets that timeout to 0, as Lucene's own TestNRTReplication.testPrimaryCloseWhileCopyingNoWait does; the test's launch() is a no-op; no socket-based job has reproduced it. That narrows the reach; it does not make installing content other than the announced CopyState intended. Verdict: a defect; the argument only narrows the window.
Recommended Carry a file over only if both jobs mean the same file
In CopyJob._transferAndCancel, carry a finished or partly copied file over only when the old job's FileMetaData for that name equals the new one (length, checksum, header, footer), +17/-2. The repro, the earlier tests, the replicator module's tests including the nightly stress test, and the static checks pass. Never salvaging across a primary change also works.
All options and trade-offs3 options · 843 words
Both fixes were built and checked in a scratch worktree at ea57360912ed. Workaround in every affected release: before promoting, make sure every replica's in-flight copy job has failed or finished (as TestStressNRTReplication does), or override transferAndCancel in your CopyJob subclass to cancel instead of transferring when the previous job's primaryGen differs. To repair an affected replica, re-copy its index from the current primary. A restart alone does not repair it (with the same primary it reopens its own commit without a sync); by reading, the first point with a higher version after a restart re-copies the stale file. The fixes were tested on main only.
Applies the module's own identity rule (name plus length, checksum, header and footer) at the one place that skipped it. Within one primary nothing changes, because a name is written once and the metadata matches; a same-named file with other content is copied again from the new primary. It also guards the in-progress branch, which today fails loudly and cancels the job; with the fix that file is copied afresh in the same job. Diffs: carry-over-same-file-only.diff, fix-with-test.diff (applies cleanly at ea57360912ed).
String prevTmpFileName = prevJob.copiedFiles.get(fileName);
// After a primary change the same file name can hold different content, so only carry a
// file over if the previous job was copying the very same file:
if (prevTmpFileName != null && isSameFile(prevJob.files.get(fileName), ent.getValue())) {
... // unchanged: keep the temp file
} else if (prevJob.current != null
&& prevJob.current.name.equals(fileName)
&& isSameFile(prevJob.current.metaData, ent.getValue())) {
... // unchanged: resume the partly copied file
}
private static boolean isSameFile(FileMetaData a, FileMetaData b) {
return a != null && b != null && a.length() == b.length() && a.checksum() == b.checksum()
&& Arrays.equals(a.header(), b.header()) && Arrays.equals(a.footer(), b.footer());
}Trade-offs
- Identical files are still salvaged across a primary change (for example files the promoted replica had also copied from the old primary); only differing ones are copied again.
- It relies on
CopyJob.filesholding the previous job's metadata for every name in itscopiedFiles, which holds for jobs built byReplicaNode.newCopyJob(read); a subclass that breaks this would re-copy (safe). - Rejected temp files are cleaned up: a finished one stays in the old job's
copiedFilesand is deleted with the others (CopyJob.java:200-202), a partly copied one is closed and force-deleted (:204-210), and the new copy writes its own uniquely named temp file (CopyOneFile.java:51-52) (read). - The fix does not repair a replica that already committed the bad state: the upgrade's restart reopens it as it is, and by reading the next point with a higher version then re-copies the stale file; re-copying the index repairs it at once. Worth a CHANGES note.
- Not reviewed by a committer.
What the checks validated
- Stock: the doc's bug test fails with assertions on and off (gradle and the harness); with the fix both tests pass (gradle
-eaand-da, harness-eaand-da). - The replicator module with the fix (on main it holds only the NRT tests): 4 classes, 17 tests, 0 failures with assertions on and off (15 skipped: the 14
@Nightlytests and the child-process helperTestSimpleServer), and with-Ptests.nightly=true17 tests, 0 failures, 1 skipped (the child-process helperTestSimpleServer);TestStressNRTReplicationran 513 s. :lucene:replicator:check -x testpasses (google-java-format, ecjLint, forbiddenApis).- Earlier hunt tests with the patched class shadowed, fresh seeds: the finder's 2, the independent reproducer's 3 (including its control) and the devil's advocate's 1 pass. The four-mode control test: the bug mode and both "nothing carried over" modes pass; mode 0 fails with and without any fix and is a separate stale-searcher lead.
In ReplicaNode.newNRTPoint, if curNRTCopy.getCopyState().primaryGen() != newPrimaryGen, call curNRTCopy.cancel(...) instead of job.transferAndCancel(curNRTCopy) (option-b.diff, +7/-1). It extends the existing "defensively discard" policy to the in-flight job.
Trade-offs
Throws away identical files too, so the first copy after a failover may re-copy more (a failover is rare). It relies on the CopyJob subclass's cancel being safe to call from newNRTPoint: the module's SimpleCopyJob makes it synchronized, the base method is not.
What the checks validated
This doc's test passes (gradle -ea; harness -ea and -da), and the earlier hunt tests give the same results as the recommended fix. The replicator module with the nightly tests and one test JVM: 17 tests, 0 failures, 1 skipped; TestStressNRTReplication ran 332 s. An earlier two-JVM nightly run under heavy machine load had 2 TestNRTReplication tests time out on a socket read during the primary's own 58 s flush; that class then passed 13 of 13 on its own. The static checks pass.
State that a controller must drain or cancel every replica's copy jobs and fence the old primary before promoting.
Trade-offs
Leaves the silent state for every application that does not, and the no-split-brain ordering shows how easy that is to miss. Not recommended.
Downstream: Lucene's replica code trusts a file name across primaries, so the mix-up happens even with authenticated transport and intact files, and a replica can keep serving a document the current primary deleted. Out of scope of SECURITY.md: no untrusted input is involved, and the module documents no guarantee for untrusted input (lines 28, 50).
Semi-formal. While writing a TLA+ model of the NRT replication protocol, an AI tool suspected from the code that the carry-over matches by name only. The TLC model checker produced an 11-step counterexample, and a JUnit replayed it on the stock jars. With the fix modelled, TLC finds no violation in the bounded configurations it checked.
The method in full242 words
Method: TLA+ model checking (TLC), a hunt unit (the NRT replication protocol, 2026-10-26). Tier semi-formal: the hypothesis came from code reading and the tool witnessed it.
- The finder was writing the
transferAndCancelstep of a TLA+ model of the protocol (NrtReplication.tla) and noticed that the carry-over compares names only. - TLC found an 11-step trace violating the invariant
NoSilentForeignContent(no replica silently serves another primary's file content): 923 distinct states with assertions off, 788 with them on. With the fix modelled as content-equality carry-over (closest to the recommended fix), TLC finds no violation in the bounded configurations it checked (978 states for the core model, 82,805 for the full one), nor under the stress test's kill-and-drain discipline (220). - A JUnit replayed the trace step by step on the stock main jars and failed; it also failed on 10.5.2 and 9.12.3, as did the independent reproducer's test; the devil's advocate's no-split-brain variant failed on main and 10.5.2.
- Then: a review by an independent reviewer from another model family (OpenAI GPT-5.6, reading only: real, medium) and a refute review by the same (procedural "not real, low": it could not run code and agreed on the root cause); an independent reproducer writing their own test in another package; a four-mode control test; a devil's advocate who found the no-split-brain ordering; an adjudication; a release sweep, a prior-report search, a security screen and a credibility audit; and this doc's own test and fix validation.
No matching public report found (2026-10-26): scripted prior-report search runs over GitHub, commits, downstream repositories, the dev@, java-user@, issues@ and commits@ lists, JIRA and the web, about 70 GitHub GraphQL searches, every replicator issue and PR, and every open PR's files. Context: LUCENE-5438 (#6501) and #14362/#14417.
Searches5 searches
Verdict new: of 21 hits read, none describes a replica keeping a temp file finished for the old primary under a name the new primary reuses. Context: #6501 / LUCENE-5438 (the original design: names can be reused after a promotion; a file is the same only if name, header and footer match), #14362 and #14417 (IndexWriter#advanceSegmentInfosCounter, against new-segment name reuse after a promotion; they do not cover delete or doc-values generation names and do not touch CopyJob). Unrelated: #16838 (file-name validation in getFilesToCopy), #13044/#13045 (configurable commit data). Downstream, nrtsearch #533 and #961 work around reused names in their own searcher manager: a stale SegmentReader is reused while the files on disk are right, a different root cause that matches a separate lead (below), not this bug. No open PR touches CopyJob.java. Limits: keyword search; GitHub's search rate limit was saturated, so GraphQL covered the same queries; the SOLR JIRA project and forks other than nrtsearch were not searched; private security reports are not visible.
Separate lead, not part of this bug: if the replica has already finished the old primary's point when it installs the new primary's point with a same-named .liv, the committed documents are right but the searcher is stale ([y0, y1] instead of [x0, x1]); it fails with both fixes too.
| Where | When | How | Found |
|---|---|---|---|
| apache/lucene issues, PRs and commits (migrated LUCENE-NNNN included), all of GitHub | 2026-10-26 | a scripted prior-report search (12 issue/PR and 2 commit queries), about 70 GraphQL searches by class, method and symptom; all 32 module:replicator items | #6501, #14362, #14417, #16838, #13044/#13045 and other replicator items; none is this bug |
| Open PRs | 2026-10-26 | the changed files of all 413 open apache/lucene PRs; the diffs of the 5 that touch lucene/replicator | none touches CopyJob.java or changes transferAndCancel, newNRTPoint or maybeNewPrimary |
| Downstream | 2026-10-26 | 2 tool queries over Elasticsearch, OpenSearch and Solr; 12 GraphQL searches and the source tree of nrtsearch | nrtsearch #533, #961 (a different root cause); the others use their own replication |
| Mailing lists and JIRA | 2026-10-26 | 32 tool list queries (dev@, java-user@, issues@, commits@), a second run with 38 queries, 18 direct lists.apache.org searches, 6 JQL queries; all comments of LUCENE-5438 | two java-user threads on NRT replication, neither about this |
| The web | 2026-10-26 | 6 queries, top 10 results each, and the feature author's 2017 design post | the design post says stale partly copied files from the old primary are removed on a cutover (supports the expected behaviour) |
Present since NRT replication was added (LUCENE-5438, 1ae7291429b, 2016; first released in 6.0.0) and unchanged in meaning since. Executed on 12 releases from 6.0.0 to 10.5.2 and on main: all fail the same way; 5.5.x lacks the API. Not a regression; not fixed upstream.
Versions tested8 rows
Introduced by 1ae7291429b ("current patch", @mikemccand, 2016-01-24), the commit that added the whole org.apache.lucene.replicator.nrt package on the LUCENE-5438 feature branch (migrated as #6501), merged into master by 12b8721a44d (2016-02-11); no pull request, no CHANGES entry. 9afa560054a (2016-02-08, same branch) added the defensive discard on a primary change for pre-copied merge files only. The carry-over loop has had only non-semantic edits since. In 6.0.0 to 6.1.0 a separate constructor bug (a missing success = true, fixed in 6.2.0) stops a plain ReplicaNode subclass from starting; with a close() override like the module's SimpleReplicaNode, the bug reproduces there too. All 83 release tags from 6.0.0 to 10.5.2 in the local history clone contain the merge (the clone lacks the 8.8.2 to 8.11.x tags; 8.11.4 was run from its release jars), so the releases not run are expected to be affected. Node was package-private, without getDirectory()/getSearcherManager(), until 6.2.0 (3daa82c6595); the history runs use a test in Lucene's own package. The fixes were tested on main only. Release jars were run on JDK 25 only.
| Version or commit | Verdict | Result |
|---|---|---|
main ea57360912ed | affected | this doc's test fails (gradle and harness, -ea and -da); the finder's 2/2; the control passes |
| 10.0.0-10.5.2 | affected | 10.0.0 and 10.5.2 executed; this doc's test fails on 10.5.2 (-ea, -da) |
| 9.0.0-9.12.3 | affected | 9.0.0 (-ea) and 9.12.3 (-ea, -da) executed |
| 7.0.0-8.11.4 | affected | 7.0.0, 7.7.3, 8.0.0 and 8.11.4 executed |
| 6.2.0-6.6.6 | affected | 6.2.0 (-ea, -da) and 6.6.6 executed, no workaround |
| 6.0.0-6.1.0 | with a close() override | reproduced (6.0.0 -ea and -da, 6.1.0) once a separate constructor bug is worked around as the module's own test node does |
| 5.5.0, 5.5.5 | no API | no nrt package; the test does not compile |
| fix (same file only) on main | not affected | this doc's test and the replicator module, including the nightly tests, pass |
An independent reproducer, a devil's advocate and an adjudication each re-ran tests on main and said real; a review from another model family (OpenAI GPT-5.6, reading only) said real, and its refute review said "not real" only procedurally. A credibility audit re-ran it with fresh seeds; this doc's test was re-run on main, 10.5.2 and with both fixes.
Checks and verdicts650 words
Every check ran on main ea57360912ed; the GPT-5.6 reviews could read code but not run it, and a separate run re-ran the repro (it failed). After the doc was written, an independent fact check, a cross-model fact check and a diagram check reviewed it.
| Check | Verdict | What it found |
|---|---|---|
| finder | candidate | TLC counterexample, then a JUnit: 2/2 fail on main (-ea, -da), 10.5.2 and 9.12.3; 10 of 10 iterations; pass with the fix sketch. |
| intended-lens review (GPT-5.6, reading only) | real, medium | Intended-behaviour argument moderate; root cause and inputs agreed. |
| Independent reproducer | real | Own JUnit in another package (public and protected API only): 2 of 3 fail on main, 10.5.2 and 9.12.3 (-ea, -da), including a no-split-brain ordering; the replica's .liv checksum equals the old primary's; the control passes; a metadata-comparing fix passes. |
| Refute review (GPT-5.6, reading only) and control test | not real, low (procedural) | Could not run code; agreed on the root cause. A separate four-mode control test: the bug mode fails and passes with the fix; the two "nothing carried over" modes pass; mode 0 is a separate lead. |
| Devil's advocate | real | No intended or misuse argument survived; its no-split-brain variant fails on main and 10.5.2 and passes with the fix. |
| adjudication | confirmed, high | Re-ran the independent reproduction's test (2 of 3 fail, -ea and -da) and the intended-behaviour check's variant (fails; passes with the fix). |
| credibility audit | keep | Re-ran the finder's test and the intended-behaviour check's variant with fresh seeds; found by reading why the stale file persists (the replica's metadata cache). |
| doc re-runs (2026-10-26) | reproduced; fixes validated | This doc's test fails on main (gradle and harness, -ea and -da) and 10.5.2; both fixes pass it and the earlier tests; the recommended fix passes the replicator module including the nightly stress test. |
| Fact check of this doc (independent AI tool) | real; 2 precision errors, fixed | Re-ran the doc's gradle command on a clean copy of ea57360912ed (1 of 2 fails, assertions on and off, word for word), the repro harness with fresh seeds on main and 10.5.2, the recommended fix (the test, the module: 17 tests, 0 failures, and the nightly stress test, 336 s) and git apply --check of all three diffs; about 60 code references correct. Fixed: the run matrix of the earlier tests, and the 15 skipped tests (14 @Nightly plus the helper TestSimpleServer); 7 minor items taken. |
| Cross-model fact check (OpenAI GPT-5.6, reading only) | real; 5 errors (2 high), fixed | Mechanism, repro output and history verified. Fixed: "a restart repairs it" (with the same primary the replica reopens its bad commit; only a later point re-copies the file), the run matrix, the in-progress branch (cancelled, not re-copied automatically), "83 release tags" (of the local clone) and the skip count; its 8 overclaims and 7 missing caveats were taken (narrow timing, the crash modelled as a close without waiting, the held-open copy job, package access, the bounded TLC result). Intended-behaviour case: moderate. |
| Diagram check | 1 text error, fixed and re-rendered | Every value, file name, checksum and the order of events matched the code and the probe logs. Fixed: the diagram's API lines wrote new PrimaryNode(...) for an abstract class (now the probe's subclass InProcPrimaryNode); also applied: the code-box range 133-157, a gap between two chips, and on the summary card the fix size (+17/-2) and the reach line (run on 12 releases and main). |
| Correction to the registry row | 2 wordings out of date | The registry row's verification (shown in the registry card) says the history sweep executed "13 releases from 6.0.0 to 10.5.2 plus main": the runs cover 12 releases plus main (history-runs/). Its notes say the in-progress branch re-copies the file and that no record shows the replicator test suite run with the fix; the branch cancels the job. |
The diagram as a still image, generated with AI and checked against the code. It is under 10 MB, the limit for a GitHub attachment.
- PNG 460 KB
Finding record, Lucene and upstream sources, repro files and diagram notes1,632 words
- Registry title
- After a primary change, an NRT replica keeps a file it already copied from the old primary when the new primary's file has the same name, matching by name only, so the replica serves and commits the wrong documents with CheckIndex clean
- Area
- replicator / NRT segment replication (CopyJob.transferAndCancel, ReplicaNode.newNRTPoint)
- Priority
- Medium, rank 73: Since 6.0.0 a failover while a replica's copy job from the old primary has finished a file that the new primary reuses by name (such as a shared segment's .liv) makes lucene-replicator NRT replication keep the old file, so the replica serves and commits the wrong deletes with CheckIndex clean and a later promotion makes that authoritative, severe silent corruption but only for opt-in NRT replication users and a rare failover timing that Lucene's own stress test never produces.
- Impact class
- data corruption
- Impact note
- Applications that use lucene-replicator's NRT segment replication (PrimaryNode/ReplicaNode/CopyJob subclasses, opt-in and @lucene.experimental; the one downstream user found, nrtsearch, inherits the code, by reading) and fail over, at any index size and in every release since 6.0.0, are hit when a replica learns of the new primary while its copy job from the old primary is still alive and has finished a file that the new primary reuses under the same name with other content (in every executed run a per-segment .liv file of a segment both primaries share, with the same delete count): the replica reports the new primary's NRT version but serves and commits a mix of both primaries' deletes (a document the new primary deleted is visible, a live one is hidden), hit counts can still match, CheckIndex reports the index clean, and if that replica is promoted later the mix becomes the authoritative index (inferred, not executed).
- Summary
- You run Lucene's NRT segment replication with failover: replica A is promoted to primary (a higher primaryGen) while replica B is still copying an NRT point from the old primary and has already finished one of its files, for example _1_1.liv. You expect B, once it has installed the new primary's NRT point, to search exactly the new primary's documents, because the module discards pre-copied data on a primary change and treats two files as the same only if name, header and footer match. Instead CopyJob._transferAndCancel keeps the old primary's finished temp file because the new primary's file has the same name, so B, at the new primary's version, shows [a0, b1] instead of [a0, a1], commits that state, and CheckIndex reports the index clean.
- Introduced
- 1ae7291429bad742715344f86cfa5200229b3698 ('current patch' by mikemccand, 2016-01-24, on the LUCENE-5438 feature branch; the commit that added the whole org.apache.lucene.replicator.nrt package, including the carry-over-by-name loop in CopyJob._transferAndCancel and the unconditional job.transferAndCancel(curNRTCopy) in ReplicaNode.newNRTPoint), merged into master by 12b8721a44db (2016-02-11); LUCENE-5438 'add near-real-time replication' (now #6501), no pull request, no CHANGES entry [read: git log --diff-filter=A, git log -S, git log -L and git blame -w in the full-history clone]; 9afa560054a (2016-02-08, same branch) later added the primary-change discard of pendingMergeFiles without covering the in-flight job (2016-01-24 (branch commit); 2016-02-11 (merge into master))
- First affected version
- 6.0.0 [exec] (release tag 2016-04-07; the first release with the org.apache.lucene.replicator.nrt package): it reproduces with a ReplicaNode subclass whose close() override skips the call made from ReplicaNode's own constructor, as the module's SimpleReplicaNode test node does, because a separate 6.0.0-6.1.0 constructor bug (success = true missing) stops a plain subclass from starting; 6.2.0 is the first release that reproduces with no workaround [exec]; 5.5.0 and 5.5.5 have no nrt package (the repro does not compile) [exec]
- Affected versions
- every release from 6.0.0 through 10.5.2 and main ea57360912ed. [exec] 6.0.0 (-ea and -da, with the close() override), 6.1.0 (with the override), 6.2.0 (-ea and -da), 6.6.6, 7.0.0, 7.7.3, 8.0.0, 8.11.4, 9.0.0, 9.12.3 (-ea and -da), 10.0.0, 10.5.2 (-ea and -da) and main (-ea and -da): both finder tests fail with the identical assertion on every one (replica at the new primary's version shows [a0, b1] instead of [a0, a1], carried-over file _1_1.liv, CheckIndex clean); the independent reproduction's independent test also fails on 10.5.2 and 9.12.3 (-ea and -da) and the intended-behaviour check's no-split-brain variant on 10.5.2 (-da). [read] all 83 release tags from 6.0.0 to 10.5.2 in the history clone contain the merge 12b8721a44db, and _transferAndCancel has had only non-semantic edits since 1ae7291429b. Not fixed on main [read: CopyJob, ReplicaNode, PrimaryNode, Node and CopyOneFile are unchanged between the study base 7cecf795236c and ea57360912ed; no open PR touches CopyJob.java]. Not affected: 5.x (API absent) [exec]. All runs on aarch64 with JDK 25.0.3.
- Status
- documented (brief, checks and a per-finding doc exist)
- Verification
- Confirmed by AI-run checks: an independent repro, a cross-model review that could only read the code (its low-confidence “not real” is procedural), an intended-behaviour check, a prior-report search and a skeptical audit; not yet triaged by a person.
- Found by
- semi-formal: TLA+/TLC, code reading
- Upstream and prior reports
- none found (2026-10-26, read-only; the prior-report record: a scripted prior-report search with 12 GitHub issue/PR queries, 2 commit queries, 2 downstream queries over Elasticsearch, OpenSearch and Solr, 32 mailing-list queries over dev@, java-user@, issues@ and commits@, 6 JIRA-archive queries and 6 web queries; a second scripted prior-report search over lists and JIRA with 38 queries and 228 hits, 0 high overlap; the finder's run with 35 queries and 23 hits, 0 high overlap; about 70 read-only GitHub GraphQL searches over apache/lucene, nrtsearch, OpenSearch, Elasticsearch, Solr and all of GitHub; every module:replicator issue and PR (32); the file lists of all 413 open apache/lucene PRs; direct lists.apache.org searches; local git history; the independent reproduction and the intended-behaviour check found nothing either); not fixed upstream (main ea57360912ed; the carry-over lines are unchanged since 1ae7291429b; no open PR touches CopyJob.java, and none of the 5 open PRs touching lucene/replicator changes transferAndCancel, newNRTPoint or maybeNewPrimary). Related context, not reports of this bug: #6501 / LUCENE-5438 (the original feature; its author notes that file names can be reused after a promotion and defines a file as the same across nodes only if name, full header and footer match), #14362 and #14417 (merged 2025-04: IndexWriter#advanceSegmentInfosCounter against segment-name reuse after promotion; it does not cover delete or doc-values generation names such as _1_1.liv and does not touch CopyJob), #16838 (open: file-name validation in ReplicaNode.getFilesToCopy), #13044 / #13045 (open: configurable commitData), and downstream nrtsearch #533 and #961 (workarounds in their own searcher manager for reused segment and generation names after a primary restart; a different root cause, SegmentReader reuse, whose fix would not fix this one)
- Lucene commit tested
- main
ea57360912ed(runs on 2026-10-26): this doc's test through gradle with assertions on and off and through the repro harness, also on the 10.5.2 release jars; the finder's test on 12 releases from 6.0.0 to 10.5.2 and on main; the recommended fix in a scratch worktree atea57360912ed(removed afterwards): this doc's test, the replicator module's tests with assertions on and off and with the nightly tests (TestStressNRTReplicationran 513 s), and the module's static checks; the alternative fix with this doc's test and the nightly module tests; every earlier hunt test with each fix shadowed.
- Finding record
lucene/replicator/src/java/org/apache/lucene/replicator/nrt/CopyJob.java:35 (@lucene.experimental), :97-112 (transferAndCancel javadoc 'Transfers whatever tmp files were already copied in this previous job and cancels the previous job'), :119-123 (skip if prevJob.exc != null), :128-156 (carry-over: prevJob.copiedFiles.get(fileName) != null moves the temp file with no FileMetaData comparison), :157-193 (in-progress branch, also matched by name)lucene/replicator/src/java/org/apache/lucene/replicator/nrt/ReplicaNode.java:61 (@lucene.experimental), :502-507 (newCopyJob javadoc: salvage files 'shared with the new copy job'), :530-545 (newNRTPoint; comment on discarding pre-copied data on a primary change, 'a bit risky to rely solely on checksum/file length'; maybeNewPrimary), :613-616 (job.transferAndCancel(curNRTCopy) whatever the primaryGen), :849-873 (fileIsIdentical: header and footer must match; 'Segment name was reused! This is rare but possible and otherwise devastating')lucene/replicator/src/java/org/apache/lucene/replicator/nrt/FileMetaData.java:20-23 (javadoc: header and footer must be identical between primary and replica to consider the files equal)lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90LiveDocsFormat.java:86-92 (index header check: segment id and generation suffix), :98 (footer check), :123-126 (delete-count check)
- The finding in 30 seconds
Lucene main ea57360912ed: lucene/replicator/src/java/org/apache/lucene/replicator/nrt/ReplicaNode.java:530-545 (newNRTPoint; comment :536-544; maybeNewPrimary :545), 581-596 (newCopyJob call), 613-616 (transferAndCancel), 706-728 (maybeNewPrimary), 834-847 (getFilesToCopy), 849-873 (fileIsIdentical), 189-206 and 301-318 (start: sync only if the primary changed), 367-373 (commit records lastPrimaryGen), 425-426 (failed job), 430-475 (finishNRTCopy; lastFileMetaData :475)Lucene main ea57360912ed: lucene/replicator/src/java/org/apache/lucene/replicator/nrt/CopyJob.java:97-112 (transferAndCancel), 114-212 (_transferAndCancel: exc check :119-123, carry-over :133-156, in-progress :157-193)Lucene main ea57360912ed: lucene/replicator/src/java/org/apache/lucene/replicator/nrt/CopyOneFile.java:74-85 (copy constructor keeps the old metaData), 99-126 (checksum checks); Node.java:196-205 (readLocalFileMetaData cache)Lucene main ea57360912ed: lucene/replicator/src/test/org/apache/lucene/replicator/nrt/Jobs.java:76-96 (the reference job runner cancels a failed job and does not requeue it)Lucene main ea57360912ed: lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90LiveDocsFormat.java:86-92 (header: segment id and generation), 98 (footer), 123-126 (delete count)
- Who is affected
Lucene main ea57360912ed: lucene/replicator/src/java/org/apache/lucene/replicator/nrt/ReplicaNode.java:61, CopyJob.java:35 (@lucene.experimental)Lucene main ea57360912ed: lucene/replicator/src/test/org/apache/lucene/replicator/nrt/SimpleCopyJob.java:184 (synchronized visit); SimpleReplicaNode.java:222-234 (wait while copying); TestStressNRTReplication.java:387-430 (promoteReplica), 1118-1119 (hit-count oracle)
- Why it looks unintended
Lucene main ea57360912ed: lucene/replicator/src/java/org/apache/lucene/replicator/nrt/ReplicaNode.java:502-507, 536-544, 849-873; FileMetaData.java:20-23; PrimaryNode.java:116-122; CopyJob.java:97-100Lucene main ea57360912ed: lucene/replicator/src/test/org/apache/lucene/replicator/nrt/TestStressNRTReplication.java:393-397, 1118-1119; SimpleReplicaNode.java:226-229Lucene main ea57360912ed: lucene/replicator/src/java/org/apache/lucene/replicator/nrt/PrimaryNode.java:207-216 (setRemoteCloseTimeoutMs: -1 = forever is the default), 339-359 (waitForAllRemotesToClose); lucene/replicator/src/test/org/apache/lucene/replicator/nrt/TestNRTReplication.java:803, 828 (testPrimaryCloseWhileCopyingNoWait sets 0)
- Suggested fix
Lucene main ea57360912ed: lucene/replicator/src/test/org/apache/lucene/replicator/nrt/SimpleCopyJob.java:240-246 (synchronized cancel); Lucene main ea57360912ed: lucene/replicator/src/java/org/apache/lucene/replicator/nrt/CopyJob.java:225-258 (cancel, not synchronized)
- TestNRTCopyAcrossPrimaryChange.java (JUnit test):
repro/LFV-241/src/org/apache/lucene/replicator/nrt/TestNRTCopyAcrossPrimaryChange.java(243 lines)
Made in parallel with this doc from the no-split-brain failover of the hunt repro; every value on screen comes from an executed probe on main ea57360912ed, from the code at that commit or from the recommended diff. Re-rendered after the diagram check (the API lines now name the probe's PrimaryNode subclass). The PNG is the final summary card.
assets/LFV-241/bug.png(460 KB)