HnswBitVectorsFormat: one float vector document closes the IndexWriter and rolls back everything since the last commit

HnswBitVectorsFormat (lucene-codecs, @lucene.experimental) indexes byte vectors only. Its writer rejects any other encoding in addField with IllegalArgumentException("HnswBitVectorsFormat only supports BYTE encoding"). IndexingChain#initializeFieldInfo calls addField the first time a vector field appears in a segment, after it has registered the field's FieldInfo, inside catch (Throwable th) { onAbortingException(th); throw th; }. So the per-document rejection is also recorded as an aborting exception: the in-memory segment is aborted, the exception becomes the writer's tragic exception, and IndexWriter rolls back to the last commit and closes itself. The caller receives the IllegalArgumentException it expected; the next addDocument or commit throws AlreadyClosedException. Buffered documents, documents already visible to near-real-time search and documents that other threads added successfully are all lost, and a fresh writer closes again if the application re-sends the document. The committed index stays intact and CheckIndex-clean.

The same format's other per-field limit, getMaxDimensions, is checked before the field is registered and is an ordinary rejection; the format's own test expects only the exception. Since 9.11.0, in all 19 releases that ship the format, and on main. A small pre-registration hook on KnnVectorsFormat fixes it and passes the codecs module's tests.

• Path. IndexWriter.addDocument (IndexWriter.java:1492) -> updateDocument (:1912-1915, call at :1913) -> the private updateDocuments (:1625-1641) -> DocumentsWriter.updateDocuments (DocumentsWriter.java:428) -> DocumentsWriterPerThread.updateDocuments (DocumentsWriterPerThread.java:291) -> IndexingChain.processDocument (:641) -> initAndValidateFields (:685-699): the field is new in this segment, so initializeFieldInfo runs (:693, :1372).
• Before registration, not aborting. validateMaxVectorDimension(..., getMaxDimensions(pf.fieldName)) (IndexingChain.java:1386-1391, plain throw at :1625).
• Registration. fieldInfos.add(new FieldInfo(...)) (:1392-1413), which also checks the schema against the global field numbers (a field already registered as BYTE fails here, not aborting), then pf.setFieldInfo(fi) (:1414).
• The aborting step. try { pf.knnFieldVectorsWriter = vectorValuesConsumer.addField(fi); } catch (Throwable th) { onAbortingException(th); throw th; } (:1445-1451). VectorValuesConsumer.addField (:68-71) creates the segment's vectors writer on first use (:52-66); PerFieldKnnVectorsFormat$FieldsWriter.addField (:118-121) picks the field's format, creates its writer on first use and puts the per-field attributes on the registered FieldInfo (:147-185). For the bit format, fieldsWriter (HnswBitVectorsFormat.java:139-148) builds the HNSW and flat writers, which open their output files.
• The throw. FlatBitVectorsWriter#addField rejects any non-BYTE encoding before it delegates (HnswBitVectorsFormat.java:193-198, throw at :195). Executed stack: FlatBitVectorsWriter.addField(:195) <- PerFieldKnnVectorsFormat$FieldsWriter.addField(:120) <- VectorValuesConsumer.addField(:70) <- IndexingChain.initializeFieldInfo(:1447) <- initAndValidateFields(:693).
• The close. IndexingChain.onAbortingException (:176-180) records it in the DWPT (DocumentsWriterPerThread.java:56-60); maybeAbort("updateDocuments", ...) (:310, :594-605) aborts the DWPT and fires onTragicEvent (IndexWriter.java:6068-6084); IndexWriter.updateDocuments' catch (:1632-1640) calls maybeCloseOnTragicEvent (:6098-6106) -> rollbackInternal(), which closes the writer without committing (rollback() javadoc, :2566-2571). The IllegalArgumentException is rethrown to the caller.

The aborting catch is defensible for codec failures that may already have changed state: by then the field is registered, the segment's vectors writer may exist with open files, and the FieldInfo carries per-field attributes. The defect is that a deterministic input check, which needs only the field name and its encoding, sits inside that catch instead of next to the dimension check.

• Who is hit. Applications on 9.11.0 or later that configure HnswBitVectorsFormat, codec-wide or per field through PerFieldKnnVectorsFormat, and send it a KnnFloatVectorField (on main also a KnnFloat16VectorField) whose field name is not yet registered as BYTE: a new field, or a field committed as FLOAT32 under another format and routed to the bit format in a later session. The format is opt-in (lucene-codecs, not the default codec) and @lucene.experimental. Any index size; one document.
• What goes wrong. Every later indexing or commit call on the writer throws AlreadyClosedException until the application opens a new one (getTragicException(), isOpen(), close() and rollback() still answer), and every document since the last commit is lost: buffered documents, documents an NRT reader already showed, and a document that a second thread had added successfully (measured; in-flight documents of a concurrent thread were not measured). The caller's IllegalArgumentException looks like an ordinary per-document rejection, so a caller that catches it and moves on finds out only on its next indexing or commit call.
• Repeats. The rolled-back segment takes the field registration with it, so re-sending the same document to a fresh writer closes that writer too.
• Not affected. The committed index stays intact and CheckIndex-clean; nothing is exposed. A field already registered as BYTE (earlier in the session or in a committed segment) gets an ordinary schema rejection and the writer stays open.
• How likely. Low: it takes the opt-in, experimental format plus a float field sent to it by mistake, and that mistake usually shows on the first such document in testing. Elasticsearch ships its own bit-vector format; no use of Lucene's class was found in Solr, OpenSearch or the OpenSearch k-NN plugin (code search of default branches, not exhaustive).
• Workaround. Send the format only KnnByteVectorField values (check the element type before addDocument). Once the field is committed as BYTE, a float document is an ordinary rejection.

Copy the test into lucene/codecs/src/test/org/apache/lucene/codecs/bitvectors/ of a Lucene checkout and run the command below. TestUtil.alwaysKnnVectorsFormat sends every vector field to the bit format; the config is the default one. The test asserts the root effect only: after the expected rejection, getTragicException() must be null. The closed writer and the rollback follow from it (the index ends with 1 document instead of the expected 3, an NRT-visible document lost, in the probe numbers above).

Under gradle it fails with -Ptests.asserts=true and false on main and passes with the fix; on the standalone runner it fails on main (seeds DEADBEEF and 5EED1234), 10.5.2 and 10.6.0, with assertions on and off. TestUtil.alwaysKnnVectorsFormat is missing on 9.11.x, 9.12.x and 10.0.0; there a variant with a FilterCodec over the default codec, same body and assertion, fails the same way on 9.11.0, 9.12.3 and 10.0.0 (9.11.1 ran only the study's own portable tests, which fail too).

cp $REPRO/TestBitVectorsRejectedFloatField.java $LUCENE/lucene/codecs/src/test/org/apache/lucene/codecs/bitvectors/
cd $LUCENE && ./gradlew :lucene:codecs:test --tests org.apache.lucene.codecs.bitvectors.TestBitVectorsRejectedFloatField -Ptests.seed=DEADBEEF -Ptests.asserts=true
# and once more with -Ptests.asserts=false: it fails both ways

# gradle, fresh worktree at main 645d4fe14a20, -Ptests.seed=DEADBEEF -Ptests.asserts=true (the same with false)
TestBitVectorsRejectedFloatField > testRejectedFloatVectorDoesNotCloseWriter FAILED
    java.lang.AssertionError: expected null, but was:<java.lang.IllegalArgumentException: HnswBitVectorsFormat only supports BYTE encoding>
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:B8FD3E78AD3F65BB]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.failNotNull(Assert.java:756)
        at org.junit.Assert.assertNull(Assert.java:738)
        at org.junit.Assert.assertNull(Assert.java:748)
        at org.apache.lucene.codecs.bitvectors.TestBitVectorsRejectedFloatField.testRejectedFloatVectorDoesNotCloseWriter(TestBitVectorsRejectedFloatField.java:39)

# probe without the test framework, abridged (main 645d4fe14a20, -ea and -da; the same on 10.6.0, 10.5.2 and 9.11.0 with -da)
A (claim) float vector {1, 2} in a new field 'bits_f' routed to HnswBitVectorsFormat
    doc 3 addDocument threw: IllegalArgumentException: HnswBitVectorsFormat only supports BYTE encoding
    getTragicException(): IllegalArgumentException: HnswBitVectorsFormat only supports BYTE encoding
    isOpen() after: false
    add doc 4 + commit: AlreadyClosedException
    NRT numDocs before doc 3: 2
    committed numDocs at the end: 1
    CheckIndex clean: true
B (control) float vector of 1025 dims in the same new field 'bits_f' (getMaxDimensions = 1024)
    doc 3 addDocument threw: IllegalArgumentException: Field [bits_f] vector's dimensions must be <= [1024]; got 1025
    getTragicException(): null
    isOpen() after: true
    add doc 4 + commit: ok
    committed numDocs at the end: 3

• The format's test. TestHnswBitVectorsFormat#testFloatVectorFails (TestHnswBitVectorsFormat.java:54-63) adds a 4-dimension KnnFloatVectorField through IndexWriter.addDocument and expects an IllegalArgumentException. It checks nothing about the writer, and its message check at :61 is a no-op (contains is not asserted).
• #12436 (2023). jpountz first suggested a dimension check inside KnnVectorsWriter#addField; mayya-sharipova objected that by then the FieldInfo is already in the global FieldInfos (review comment r1275409313); jpountz then liked "retaining the property that the field info doesn't even get created if its number of dimensions is above the limit" (r1275939419), and uschindler welcomed that the check runs in addDocument() rather than while indexing or flushing. That thread named the poisoned-FieldInfo hazard, not the writer closing, and predates the bit format.
• Lucene's tests. TestIndexWriterExceptions#testNullStoredField (:1592-1614, "test a null string value doesn't abort the entire segment") asserts getTragicException() is null after an input IllegalArgumentException and that the earlier document survives. BaseKnnVectorsFormatTestCase#testIllegalDimensionTooLarge (:779-795) and TestPerFieldKnnVectorsFormat#testMaxDimensionsPerFieldFormat (:217-244) keep using the writer after a codec's per-field rejection.
• Precedent. LUCENE-5662 (#6724, Robert Muir): missing null checks in Field aborted segments, "This is bad because you lose other unrelated documents in IndexWriter's buffer"; fixed under "Bug fixes" in 4.8.1 (CHANGES.txt:12384-12385). LUCENE-5611 made doc-values and term-vector setting errors non-aborting. LUCENE-5987 defines an aborting exception as "an exception that would otherwise cause silent data loss" (CHANGES.txt:11577-11579).
• Nothing documents the close. The IndexWriter class javadoc names "an Error, or disaster strikes during a checkpoint" (IndexWriter.java:162-165; not a complete list); addDocument says that after an Exception "the index will be consistent, but this document may not have been added" (:1465-1466); getTragicException gives "disk full while flushing a new segment" as its example (:6108-6112). No javadoc, comment at the catch, CHANGES or MIGRATE entry says this rejection closes the writer.
• History. The aborting catch came with #992 (LUCENE-10592, 2022), with no comment about input checks. In the #13288 review the author said the wrapper writer exists because "it validates only BYTE vectors are written" and jimczi replied "Ah thanks, I missed the addField validation": meant as validation; nobody discussed that it aborts.

The case (from the cross-model reviewer and the study's devil's advocate): a float vector is outside the domain of a byte-only, experimental format, so this is application misconfiguration. On 2024-06-05, asked on #13288 whether the format "was more a prototype and not intended for production use", its author answered yes. IndexingChain marks every Throwable from addField aborting on purpose, and addDocument promises a consistent index, not an open writer; the commit does survive.

Why it fails: the format answers this input with a deliberate IllegalArgumentException that its own test expects from addDocument, so the input is handled, not undefined; the problem is that the rejection also discards unrelated documents, the case LUCENE-5662 fixed as a bug. It does not merely fail loudly: the bad call gets its exception, but the documents discarded with it were added by calls that had already returned successfully, and those calls never report an error. The aborting catch protects against codec failures that may already have changed state, and the same IndexingChain checks the codec's dimension limit before registration: it was placed there in #12436 so that no FieldInfo is created for a rejected field and so that the check runs in addDocument rather than during indexing or flush, and that placement is why that rejection does not abort. @lucene.experimental marks an unstable API, not permission to lose writes. The argument lowers the impact and makes an "improvement" label likely; it does not make the close intended.

Four options; A was validated with gradle in a fresh scratch worktree at 645d4fe14a20. B and D were not executed; C is the study's root-cause demonstration, not a patch. With A the caller sees the same exception and message in the bug case; only the writer state changes. For a field already registered as BYTE in an earlier segment, the format's message replaces today's schema-consistency message (cannot change field ... from ... BYTE ... to inconsistent ... FLOAT32 ...); both are non-aborting.

4 files, +31 / -3. KnnVectorsFormat gets public void checkVectorEncoding(String fieldName, VectorEncoding encoding) {} with javadoc (the default accepts every encoding, so no other format changes); PerFieldKnnVectorsFormat delegates it to getKnnVectorsFormatForField(fieldName), like getMaxDimensions; HnswBitVectorsFormat overrides it with the same IllegalArgumentException it throws today; IndexingChain#initializeFieldInfo calls it right after validateMaxVectorDimension, before fieldInfos.add. FlatBitVectorsWriter#addField keeps its check as a backstop.

// KnnVectorsFormat: the default accepts every encoding
public void checkVectorEncoding(String fieldName, VectorEncoding encoding) {}

// HnswBitVectorsFormat
@Override
public void checkVectorEncoding(String fieldName, VectorEncoding encoding) {
  if (encoding != VectorEncoding.BYTE) {
    throw new IllegalArgumentException("HnswBitVectorsFormat only supports BYTE encoding");
  }
}

// IndexingChain#initializeFieldInfo, right after validateMaxVectorDimension(...)
vectorsFormat.checkVectorEncoding(pf.fieldName, s.vectorEncoding);

Trade-offs

• API addition: a new public, non-abstract method on KnnVectorsFormat; backwards compatible and backportable to 10.x, but the maintainers must want it. A boolean supportsVectorEncoding(...) would mirror getMaxDimensions more closely, at the cost of a generic message composed in IndexingChain, which does not know the per-field format's name.
• Message: in the bug case the exception type and message stay as today. For a field already registered as BYTE in an earlier segment, HnswBitVectorsFormat only supports BYTE encoding replaces today's cannot change field "v" from ... vector encoding=BYTE ... to inconsistent ... vector encoding=FLOAT32 ...; both are non-aborting (fact-check probe, stock and with the fix).
• Wrappers: a format that wraps another and delegates getMaxDimensions must delegate this too; in Lucene's main sources that is only PerFieldKnnVectorsFormat (AssertingKnnVectorsFormat wraps the default format, which accepts everything).
• Indexing path only: FlatBitVectorsWriter#mergeOneField has no encoding check, so addIndexes or a merge that brings float segments under the bit format is a separate, untested question.
• Future formats: open PR #16092's PreconditioningKnnVectorsFormat throws from addField for non-FLOAT32 fields (read from the diff); it could override the same hook.

What the checks validated

Gradle in a fresh scratch worktree at 645d4fe14a20 (JDK 25.0.3, -Ptests.seed=DEADBEEF), 2026-10-30:

• without the fix the minimal test fails with -Ptests.asserts=true and false; git apply fix.diff succeeds; with it the test passes with assertions on and off;
• :lucene:codecs:test, the whole module: 27 suites, 694 tests, 78 skipped, 0 failures (TestHnswBitVectorsFormat 8 of 8); core codecs.perfield.*, codecs.lucene99.*, codecs.lucene104.*, TestIndexWriterExceptions*, index.TestKnn*, search.TestKnn*: 24 suites, 807 tests, 34 skipped, 0 failures;
• checkGoogleJavaFormat (core, codecs), ecjLintMain (core, codecs), ecjLintTest (codecs), forbiddenApisMain (core, codecs) and core renderJavadoc pass;
• a one-off scratch check (not a repro): after a rejected float32 and a rejected float16 document in new field v, the writer is open, a KnnByteVectorField in v is accepted, and the commit holds 3 documents with v registered as BYTE;
• after git apply -R the minimal test fails again; the worktree was removed afterwards.

Catch IllegalArgumentException from addField without calling onAbortingException. Not executed; not recommended.

Trade-offs

By then the field is registered as FLOAT32 in the segment and the global field numbers, carries per-field attributes, and the segment's vectors writer may have opened files; later BYTE documents in that field would fail the schema check for the rest of the session (the hazard named in #12436), and a blanket catch could hide real codec failures.

The study's demonstration checks the format name "HnswBitVectorsFormat" next to getMaxDimensions. With it the minimal test passes (assertions on and off) and so do the 8 TestHnswBitVectorsFormat tests.

Trade-offs

Proves the root cause, but core must not know a codecs-module format by name. Not a patch.

A javadoc note on the format. Not executed.

Trade-offs

Honest minimum, but it keeps the data loss and contradicts the format's own test.

Method: code reading, then executed tests (the hunt's non-formal tier).

• First signal (2026-10-25). A unit studying IndexingChain's schema and exception handling read the aborting / non-aborting classification while building the exception-path part of an Alloy model; no model artifact was produced for it and no formal tool witnessed it. A JUnit on the stock main jars confirmed the close and the rollback.
• Checks. An independent reproducer with its own test and controls, an executed cross-model works-as-designed review (OpenAI GPT-5.6), an intended-behaviour devil's advocate with 7 scope probes, and an adjudication confirmed it; a follow-up swept the releases, searched for prior reports and measured the scope of the loss; the repro was reduced to one test and one assertion; on 2026-10-30 a tie-breaker re-ran it on main, 10.6.0, 10.5.2 and 9.11.0 with a probe that uses no test framework.