IndexWriter#addBatch lets a rejected frozen FieldType skip the per-field schema check, so a wrong-dimension vector is stored truncated

Since 10.5.0 Lucene skips the per-field schema check for a document whose field uses a frozen FieldType instance that already passed the check in the current in-memory segment. A document that fails the check leaves its type behind as the cache's candidate, and the experimental IndexWriter#addBatch, which sets no candidate of its own before that point, promotes it after validating its own column for that field. From then until the next flush, every document with the rejected type instance skips the check: with the default codec an 8-dimensional vector is accepted into a 4-dimensional field and stored as its first 4 values, with CheckIndex clean, and a SortedDocValuesField on a NUMERIC field fails with ClassCastException instead of the schema IllegalArgumentException. The root cause came with the columnar API (#15990); the reported form needs #16219 and #16171; all ship in 10.5.0, 10.5.1, 10.5.2 and 10.6.0 and are on main. A 3-line fix in processBatch was validated.

1. The cache. Each PerField keeps validatedFrozenFieldType and candidateFieldType (IndexingChain.java:1853-1859). PerField.reset (:1876-1886), called for each field of each document (:623), first sets candidateFieldType = fieldType (:1878); for a type other than the cached one it clears the cache and the per-document schema (:1879-1885).
2. The check. A field without a cached type is collected for validation (:624-626) and its per-document schema is built (:635-637). initAndValidateFields (:685-699) runs assertSameSchema against the field's FieldInfo (:695; vector dimension at :2354) and only then trySetValidatedFrozenFieldType (:697), which caches the candidate if it is a frozen FieldType and clears it (:1892-1898).
3. The leftover. For the 8-dimensional document assertSameSchema throws, :697 is not reached, and the rejected type stays as the candidate. On the row path that is harmless: the next document's reset overwrites it before anything is promoted.
4. The promotion. processBatch (:717), on a field's first column (:758-765), resets the field's schema (:763) but not the candidate, builds the schema from the column's type (:779) and calls initAndValidateFields (:784). The valid 4-dimensional column passes assertSameSchema, and :697 caches the leftover 8-dimensional type. The batch sets no candidate of its own before this promotion (with a stored or inverted column, its row pass sets one at :914, afterwards; the repro's batch has none).
5. The skipped check. The next document with that instance matches the cache in reset (:1879), is not collected for validation (:624) and builds no schema (:635). processField (:1456-1470) dispatches on the document's own type: indexVectorValue (:1765-1779) passes the 8 values to the writer made for the 4-dimensional field, and the flat vectors writer copies the first dim of them (Lucene99HnswVectorsWriter.java:832, Lucene99FlatVectorsWriter.java:498, ArrayUtil.copyOfSubArray(value, 0, dim) at :468). For doc values, indexDocValue casts the NUMERIC writer to SortedDocValuesWriter (:1746).
6. How long. Until the in-memory segment is flushed (commit, NRT reopen or flush): a new segment starts with fresh PerField state.

A one-off copy of the minimal test that adds [1..8] and reads the vectors back prints stored: [0.0, 0.0, 0.0, 0.0][0.0, 0.0, 0.0, 0.0][1.0, 2.0, 3.0, 4.0] on 645d4fe14a20. The finder's test, re-run there through gradle: 8-dimensional vector accepted into a 4-dimensional field (seqNo=3); stored vectors: [1.0, 1.0, 1.0, 1.0] [2.0, 2.0, 2.0, 2.0] [1.0, 2.0, 3.0, 4.0] for the input [1..8]; an independent reproducer's own test found CheckIndex clean=true; FieldInfo dim=4 (on ea57360912ed). The doc-values form: ClassCastException: class org.apache.lucene.index.NumericDocValuesWriter cannot be cast to class org.apache.lucene.index.SortedDocValuesWriter, with the stock static SortedDocValuesField.TYPE; the writer stays open and has no tragic exception (the independent reproduction and the adjudication).

Controls (independent reproducers): without the addBatch, with an addBatch on another field, or with an addBatch but no earlier rejected document, the 8-dimensional document is rejected every time; after a commit it is rejected again.

• New segment. The rejected document can be the first document after a commit; the 8-dimensional vector is still accepted and truncated.
• Partial schema. A document rejected because of another field, before f has a FieldInfo, leaves f's type as the candidate; the first addBatch for f (points + SORTED_NUMERIC) caches it, and a later SORTED_NUMERIC-only document is accepted into the points + SORTED_NUMERIC field. This form exists since #15990.
• Every per-field check is skipped, not just the dimension: in the follow-up's probe an exact zero vector with a EUCLIDEAN type was accepted into a COSINE field.
• Not reachable this way: a stored or inverted column for the field in the batch re-runs PerField.reset in the row pass (:912-915), after the promotion at :784; for a column type that is a different instance from the promoted one, reset drops the just-promoted cache again (:1879-1885, validatedFrozenFieldType = null at :1883). The probe's term-vector mismatch stayed rejected.

The study's bounded-exhaustive harness (123,424 document/addBatch/document triples on ea57360912ed) found 1,384 mismatching sequences on stock. This fix alone removes all 816 wrong-exception sequences and 56 of the 276 accepted-although-invalid ones; the other 512 go with two sibling defects in the same validation state (LFV-116 and LFV-118), and 0 remain with all three fixes.

• Who. Applications on 10.5.0 and later that use the experimental IndexWriter#addBatch (or updateDocuments(Term, ColumnBatch), which reaches the same processBatch by code reading; not run) with a doc-values, points or vector column for a field they also fill through addDocument; that keep indexing after Lucene rejects a document for a per-field schema mismatch on that field (a mis-sized embedding, a vector with the wrong similarity, a mixed-up doc-values class) while its field uses a reused frozen FieldType instance (a cached KnnFloatVectorField.createFieldType result, or the stock static NumericDocValuesField.TYPE / SortedDocValuesField.TYPE); and that reuse the instance before the next flush.
• Not hit. KnnFloatVectorField(name, vector, similarity) makes a new type for every field, so it never matches the cache. Releases before 10.5.0 have no addBatch.
• What it costs. With the default codec a larger wrong-dimension vector is stored as its first dim values, with CheckIndex clean; a smaller one is rejected with ArrayIndexOutOfBoundsException instead of the schema error. Scalar-quantized vector formats (2 of the 3 probed) reject both with ArrayIndexOutOfBoundsException. Doc values: a ClassCastException instead of the schema IllegalArgumentException; the document is still rejected and the writer stays open. (In the harness, an indexed StringField type promoted into a doc-values-only field failed with a NullPointerException instead.)
• Other documents. Stored vectors of other documents never changed and the writer never closed with assertions off (probe: 300 later documents, commit, a second segment, forceMerge(1), 3 vector formats). But the similarity check is skipped too: an exact zero vector with a EUCLIDEAN type reached a COSINE field, and 19 of 100 kNN self-queries for other documents of that segment missed with the default codec (100 of 100, and a CheckIndex failure, with the scalar-quantized HNSW format). With assertions on, that format's next flush can fail with an AssertionError that closes the writer as a tragic event (test configuration only). LFV-046 (tiny-norm COSINE vectors) reaches the same effect without these preconditions.
• Conditions. Default IndexWriterConfig and codec, assertions on and off, ordinary scale: one rejected document and one one-document batch suffice. Only single-threaded runs.
• How likely: rare. It needs the experimental API on the same field as addDocument, a schema-mismatch rejection the application survives, and the same type instance again before a flush. Mapping layers that check dimensions before Lucene make the rejection itself unusual (Elasticsearch does, by code reading).

The test sits in org.apache.lucene.index, as Lucene's own tests do, but uses no package-private API. The batch is a small anonymous ColumnBatch with one sparse VectorColumn; the codec is pinned to the default codec and the directory is a ByteBuffersDirectory, so the outcome does not depend on the seed. Line 72 (the first expectThrows, before the batch) passes; only line 75 fails.

The stored-value check ([1..8] stored as [1, 2, 3, 4], CheckIndex clean) and the doc-values form are shown by the study's longer tests (in the collapsed explanation above); they are not part of this repro.

Regression test: this same test; it passes with the recommended fix.

cp $REPRO/TestAddBatchRejectedFrozenType.java $LUCENE/lucene/core/src/test/org/apache/lucene/index/
cd $LUCENE && ./gradlew :lucene:core:test --tests TestAddBatchRejectedFrozenType -Ptests.seed=DEADBEEF -Ptests.asserts=true
# assertions off fails the same way: -Ptests.asserts=false

# gradle, stock 645d4fe14a20, -Ptests.seed=DEADBEEF -Ptests.asserts=true (stack trimmed; -Ptests.asserts=false identical)
TestAddBatchRejectedFrozenType > testWrongDimensionStaysRejectedAfterAddBatch FAILED
    junit.framework.AssertionFailedError: Expected exception IllegalArgumentException but no exception was thrown
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:AAAE0EF6A92D1D49]:0)
        at app//org.apache.lucene.tests.util.LuceneTestCaseParent.expectThrows(LuceneTestCaseParent.java:1432)
        at app//org.apache.lucene.tests.util.LuceneTestCaseParent.expectThrows(LuceneTestCaseParent.java:1418)
        at app//org.apache.lucene.index.TestAddBatchRejectedFrozenType.testWrongDimensionStaysRejectedAfterAddBatch(TestAddBatchRejectedFrozenType.java:75)
1 test completed, 1 failed

# the study's longer tests on the same stock build (gradle, seed DEADBEEF): 7 tests completed, 5 failed, both controls pass
TestAddBatchCachesRejectedFieldType > testVectorOfWrongDimensionAcceptedAndTruncated
    java.lang.AssertionError: 8-dimensional vector accepted into a 4-dimensional field (seqNo=3); stored vectors: [1.0, 1.0, 1.0, 1.0] [2.0, 2.0, 2.0, 2.0] [1.0, 2.0, 3.0, 4.0]
TestAddBatchCachesRejectedFieldType > testDocValuesTypeMismatchNotReportedAsSchemaError
    junit.framework.AssertionFailedError: Unexpected exception type, expected IllegalArgumentException but got java.lang.ClassCastException: class org.apache.lucene.index.NumericDocValuesWriter cannot be cast to class org.apache.lucene.index.SortedDocValuesWriter
TestAddBatchStaleCandidateNewSegment > testVectorRejectedInNewSegmentThenBatch     same vector message (seqNo=5)
TestAddBatchStaleCandidateNewSegment > testDocValuesRejectedInNewSegmentThenBatch  same ClassCastException
TestAddBatchPromotesCandidateOfUninitializedField > testCandidateOfRejectedDocumentPromotedByFirstBatch
    java.lang.AssertionError: ACCEPTED: a SORTED_NUMERIC-only document was accepted into a points+SORTED_NUMERIC field

• One call earlier it is rejected. The repro's first expectThrows (before the batch) passes; only the batch in between changes the outcome.
• The contract. The FieldSchema javadoc: "If there is inconsistency, we raise an error. This ensures that a field has the same data structures across all documents." (IndexingChain.java:2208-2216). MIGRATE.md:1557-1564: "The per field data-structures are implicitly defined by the first document indexed that contains a certain field. Once defined, the per field data-structures are not changeable for the whole index." The KnnFloatVectorField javadoc: "Vectors of a single field share the same dimension and similarity function".
• The cache was meant to skip only redundant work. CHANGES.txt (10.5.0): "Cache frozen FieldType to skip redundant schema validation" (#15886). That commit's comment on the candidate: "This will be promoted in trySetValidatedFrozenFieldType if it is frozen and valid". A type that failed validation is neither.
• Upstream tests. BaseKnnVectorsFormatTestCase.testIllegalDimChangeTwoDocs (same segment) and TestIndexOptions.testFrozenFieldTypeTransitionAcrossDocuments (two frozen types) expect IllegalArgumentException. None of the frozen-type tests calls addBatch.
• Precedent. #16708 ("Validate vector values in columnar addBatch", merged 2026-09-28; CHANGES 10.6.0, Improvements) closed an earlier gap where addBatch accepted vectors that addDocument rejects, by adding the same checks to the column path. No discussion on #15886, #16171, #16219 or #15990 mentions rejected documents or promotion from a batch.

The strongest case: addBatch is @lucene.experimental, the wrong-dimension document is the application's own mistake, and the cache trusts type identity, so reusing a type Lucene rejected is misuse.

Why it fails: @lucene.experimental is about API stability ("This API is experimental and might change in incompatible ways in the next release"), not about relaxing checks for later addDocument calls. Lucene enforces the rule itself and rejects the identical document one call before the batch, so the input is outside what the API accepts. The doc-values form triggers with Lucene's own static TYPE constants, which every application reuses. The checkers who argued this side (a GPT-5.6 reviewer and a devil's-advocate verifier) rated it weak.

Option A was built and run. B and C are alternatives from the verifiers' notes and this brief; they were not run.

Right where processBatch starts a fresh schema for a field (IndexingChain.java:758-765), clear the cache candidate before the batch validates and promotes. This is the finder's fix, rebased onto 645d4fe14a20 with a 2-line offset.

        pf.fieldGen = batchGen;
        pf.columnFeatures = (byte) columnFeatures;
        // a column's type is not a candidate for the frozen-type cache; drop any candidate left
        // behind by an earlier document that never reached (or failed) validation
        pf.candidateFieldType = null;
        pf.schema.reset(baseDocID);

Trade-offs

• Local. One line in processBatch, which promotes without having set a candidate (the other promotion call, for the reserved parent field, never sees a candidate: reset is never called on it, and documents and batches cannot use that field). It covers the same-segment, new-segment and partial-schema forms.
• No change on the happy path, apart from at most one lost cache hit. A candidate still pending when a batch starts a field is the leftover of a document that did not pass validation, a type that is already cached, or the column type that an earlier batch's row pass recorded (a stored or inverted column, IndexingChain.java:914 -> :1878), which on stock the next batch for that field promotes at :697. Clearing it leaves validatedFrozenFieldType as it was; in the last case the next valid document with that type is checked once and then cached, as today.
• The stale state itself stays. A failed document still leaves its candidate behind; any future caller of trySetValidatedFrozenFieldType that does not set a candidate first needs the same care (options B and C).

What the checks validated

Applied to a scratch worktree of 645d4fe14a20 and run through gradle with OpenJDK 25.0.3, seed DEADBEEF (2026-10-30):

• stock: the minimal test fails with assertions on and off; the study's three longer test classes fail 5 of 7 (both controls pass);
• with the fix: the minimal test passes with assertions on and off; the three longer classes pass 7 of 7 with assertions on and off;
• the closest suites TestIndexOptions, TestColumnBatchIndexing and the six org.apache.lucene.document.column test classes pass (180 tests, 0 failures);
• the whole :lucene:core:test passes (748 classes, 8,897 tests, 0 failures, 322 skipped); checkGoogleJavaFormat, ecjLintMain and forbiddenApisMain for core pass.

In processDocument, on any exception from the first pass or initAndValidateFields (IndexingChain.java:611-642), set candidateFieldType = null on every PerField the document touched.

Trade-offs

Removes the stale state at its source, so future callers are safe too; but it must cover every throw site of the first pass (reserved field name, updateDocFieldSchema, initializeFieldInfo, assertSameSchema), including the partial-schema case where the failing field is not the poisoned one.

Pass the type that processDocument just validated to the promotion (or record the candidate only after validation), and let processBatch validate without promoting.

Trade-offs

Removes the hidden coupling between reset and the promotion that this bug came from; a larger change to code #16171 tuned for performance, so it would want a benchmark.

Method: semi-formal (the finder tagged it formal): a hypothesis from code reading, witnessed by a model checker and then by executed tests.

• First signal. Extending the unit's Alloy 6.2 model of the FieldSchema/PerField frozen-type cache with the processBatch step, the finder noticed in the code that processBatch calls trySetValidatedFrozenFieldType without setting candidateFieldType first.
• Alloy. The check BatchStaleCandidate (the row-path fixes on, the batch fix off) is satisfiable in the full and the small universe: a 3-step counterexample, a rejected document, an addBatch for the same field, then the same document accepted. With all three fixes of the unit, FixedLocal is unsatisfiable in the small universe (3 FieldTypes, 3 documents, 4 states); the same check in the full universe and a deeper small-universe bound timed out at 3000 s.
• Real jar. A bounded-exhaustive differential harness ran every (document, addBatch with a NUMERIC column, document) triple, 123,424 sequences, and found 1,384 mismatches on stock.
• JUnit. The finder's public-API test showed the silent vector truncation and the doc-values ClassCastException; independent verifiers wrote their own tests; the minimal repro above is the vector case cut down to one assertion.