InitializedHnswGraphBuilder.fromGraph allows null initializedNodes, but addGraphNode then throws NullPointerException

The finding in 30 seconds

fromGraph copies a graph into a builder; an optional bit set marks the copied nodes so that later addGraphNode calls skip them. With null there is no bit set, but addGraphNode still looks the new node up in it and throws.

Background: continuing an HNSW graph

An HNSW graph is Lucene's index for kNN vector search: each vector is a node, linked to a few nearby nodes.
InitializedHnswGraphBuilder.fromGraph copies an existing graph into a new builder, so more vectors can be inserted with addGraphNode instead of rebuilding the graph from scratch. It was written for segment merges, but Lucene's own merges now use it only to copy or repair a graph; their insertions go through other builders.
Its initializedNodes bit set marks the copied nodes, so that inserting them again is skipped.

Diagram of the bug: two vectors, node 0 at [1, 0] and node 1 at [0, 1]. An HNSW graph holding node 0 is copied with InitializedHnswGraphBuilder.fromGraph and initializedNodes = null, which the javadoc allows. addGraphNode(1) first calls initializedNodes.get(1) to see whether node 1 was copied; with null this throws NullPointerException at InitializedHnswGraphBuilder.java:489 and the graph keeps 1 node. With a bit set marking node 0, the check says node 1 is new, it is inserted and linked to node 0, and the graph has 2 nodes. — A 1-node graph is copied with no bit set of copied nodes (`null`). `addGraphNode(1)` first asks that bit set whether node 1 was copied, and throws because there is none. With a bit set marking node 0, node 1 is inserted and the graph has 2 nodes. Select the still to open it at full size. Generated with AI, checked against the code.

The finding, step by step

Take two 2-d vectors, [1, 0] and [0, 1], and build an HNSW graph of the first one only: 1 node.
Copy it into a new builder with InitializedHnswGraphBuilder.fromGraph(scorer, 100, 42L, graph, new int[] {0}, null, 2). The null is the bit set of copied nodes, which the javadoc allows: "can be null if not tracking".
Call addGraphNode(1). You expect a 2-node graph, which is what you get when you pass a bit set marking node 0 instead of null.
Instead it throws NullPointerException: Cannot invoke "org.apache.lucene.util.BitSet.get(int)" because "this.initializedNodes" is null, with assertions on and off. The graph keeps 1 node.
Why: addGraphNode first asks the bit set whether the node was copied, so it can skip it, and never checks that there is a bit set (InitializedHnswGraphBuilder.java:489). The same method in HnswConcurrentMergeBuilder's private worker class does check.

Who is affected

Only code that calls the experimental fromGraph itself with a null bit set and then inserts, such as a custom graph merger or an offline graph tool. IndexWriter, merges and search never insert through such a builder. It fails loudly on the first insertion, before the graph changes; passing a bit set of the copied nodes avoids it.

Mechanism, terms and numbers431 words

InitializedHnswGraphBuilder.fromGraph copies an existing HNSW graph into a new builder so that more vectors can be inserted. Its initializedNodes bit set marks the copied nodes, which later addGraphNode calls skip; the @param says it "can be null if not tracking" (InitializedHnswGraphBuilder.java:109-110 at 5adff78bb168), and the factory returns the builder "ready for additional operations" (:99-101). But addGraphNode starts with if (initializedNodes.get(node)) return; and no null check (:489), so with null the first insertion throws NullPointerException, with assertions on and off, before the graph changes. #15003 (first released in 10.4.0) added the "can be null" wording and the first in-tree null caller, initGraph, which never inserts, but left the older read unguarded; the same override in HnswConcurrentMergeBuilder's private ConcurrentMergeWorker already guards it (HnswConcurrentMergeBuilder.java:561). Lucene's own merges never insert through a builder created with null, so only code that uses this @lucene.experimental builder directly is hit. A one-line guard fixes addGraphNode; rejecting null and correcting the javadoc is the alternative. Priority Low.

The mechanism (main `5adff78bb168`, `lucene/core/src/java/org/apache/lucene/util/hnsw/`)

Null is documented as legal. @param initializedNodes bit set marking which nodes are already initialized (can be null if not tracking) (InitializedHnswGraphBuilder.java:109-110). Both public overloads (:120, :150) pass it unchecked to the private constructor, which stores it (:232).
The bit set is a skip list for insertion. Field javadoc: the copied nodes "will be skipped during subsequent addGraphNode(int) calls" (:58-61).
The skip check has no null check (the fix site). addGraphNode (:487-493) begins if (initializedNodes.get(node)) { return; } (:489), so a null bit set throws before super.addGraphNode(node) touches the graph. build(maxOrd) reaches the same line through addVectors (HnswGraphBuilder.java:208).
The sibling guards it. The private ConcurrentMergeWorker nested in HnswConcurrentMergeBuilder (HnswConcurrentMergeBuilder.java:493) has the same override as if (initializedNodes != null && initializedNodes.get(node)) (HnswConcurrentMergeBuilder.java:559-565).
Why Lucene itself never hits it. The in-tree null callers never insert: initGraph passes null (:203) and returns only builder.getGraph() (:206); the package-private pruneGraph (:249-270) passes null (:263) and is used for repair and rebalancing only; ConcurrentHnswMerger builds its own FixedBitSet for insertion (ConcurrentHnswMerger.java:78).

In the repro the copied graph has 1 node and room for 2. With a bit set marking node 0, addGraphNode(1) inserts node 1 and the graph has 2 nodes, linked to each other (an executed control with the repro's exact inputs, assertions on and off). With null it throws at :489 and the graph keeps 1 node.

Exposure and likelihood161 words

Who. Code that calls the public fromGraph with initializedNodes = null, as the javadoc allows, and then inserts with addGraphNode (or build). The class is public and final in an exported package (InitializedHnswGraphBuilder.java:56, module-info.java:50) and tagged @lucene.experimental (:54).
Default vs opt-in. Not on any default path: IndexWriter, serial and concurrent merges and search never insert through a builder created with null.
Effect. One NullPointerException on the first insertion, with only the JVM's null-dereference text and no Lucene error message. The read is the first statement of addGraphNode(int), so it throws before super.addGraphNode touches the graph: the graph stays as copied, and the caller gets an exception, not a wrong or partial graph.
Workaround. Pass a bit set of the copied ordinals, as Lucene's own tests do (HnswGraphTestCase.java:700-715, :737-754).
Scale and likelihood. One copied node and one insertion are enough. Rare: the documented searches found no report since 10.4.0.

Minimal repro

One JUnit test method on the public API: two 2-d vectors, a 1-node graph, a copy with a null bit set, one addGraphNode call. Deterministic and single-threaded; it fails with assertions on and off.

TestInitializedHnswGraphBuilderNullNodes.java (lucene/core, package org.apache.lucene.util.hnsw, 1 test)40 lines

/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.apache.lucene.util.hnsw;

import java.io.IOException;
import java.util.List;
import org.apache.lucene.codecs.hnsw.DefaultFlatVectorScorer;
import org.apache.lucene.index.FloatVectorValues;
import org.apache.lucene.index.VectorSimilarityFunction;
import org.apache.lucene.tests.util.LuceneTestCase;

public class TestInitializedHnswGraphBuilderNullNodes extends LuceneTestCase {

  // Expected: initializedNodes "can be null if not tracking", so node 1 is added; actual: NPE.
  public void testAddNodeWithNullInitializedNodes() throws IOException {
    RandomVectorScorerSupplier scorer =
        DefaultFlatVectorScorer.INSTANCE.getRandomVectorScorerSupplier(
            VectorSimilarityFunction.EUCLIDEAN,
            FloatVectorValues.fromFloats(List.of(new float[] {1, 0}, new float[] {0, 1}), 2));
    OnHeapHnswGraph graph = HnswGraphBuilder.create(scorer, 16, 100, 42L).build(1);
    InitializedHnswGraphBuilder builder =
        InitializedHnswGraphBuilder.fromGraph(scorer, 100, 42L, graph, new int[] {0}, null, 2);
    builder.addGraphNode(1);
    assertEquals(2, builder.getGraph().size());
  }
}

Command

cp $REPRO/TestInitializedHnswGraphBuilderNullNodes.java $LUCENE/lucene/core/src/test/org/apache/lucene/util/hnsw/
cd $LUCENE && ./gradlew :lucene:core:test --tests org.apache.lucene.util.hnsw.TestInitializedHnswGraphBuilderNullNodes -Ptests.seed=DEADBEEF
# and with -Ptests.asserts=false: the same failure

Expected: addGraphNode(1) inserts node 1 and the graph has 2 nodes.
Observed: NullPointerException: Cannot invoke "org.apache.lucene.util.BitSet.get(int)" because "this.initializedNodes" is null at InitializedHnswGraphBuilder.addGraphNode(InitializedHnswGraphBuilder.java:489).

All files of this repro: repro/LFV-1197/

More files, environment, full output142 words

TestInitializedHnswGraphBuilderNullNodes (one method, extends LuceneTestCase, package org.apache.lucene.util.hnsw of the lucene/core tests). Fixed vectors, and the builders' random level draws use the fixed seed 42L. About 0.3 s of test time. checkGoogleJavaFormat accepts the file, and the same test serves as the regression test.

Environment

lucene	`main 5adff78bb168 (12.0.0-SNAPSHOT): fails; 10.4.0, 10.5.2, 10.6.0: the same or an equivalent test fails; 9.9.0, 9.12.3 and 10.3.2 (API-adapted copies): the same exception, but there the javadoc did not allow null`
jdk	`OpenJDK 25.0.3 (build 25.0.3+9-LTS)`
os	`Linux aarch64 (nothing platform-specific)`

Command (environment)

cp $REPRO/TestInitializedHnswGraphBuilderNullNodes.java $LUCENE/lucene/core/src/test/org/apache/lucene/util/hnsw/
cd $LUCENE && ./gradlew :lucene:core:test --tests org.apache.lucene.util.hnsw.TestInitializedHnswGraphBuilderNullNodes -Ptests.seed=DEADBEEF
# and with -Ptests.asserts=false: the same failure

Observed output

TestInitializedHnswGraphBuilderNullNodes > testAddNodeWithNullInitializedNodes FAILED
    java.lang.NullPointerException: Cannot invoke "org.apache.lucene.util.BitSet.get(int)" because "this.initializedNodes" is null
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:5902F0C99CD7A4FE]:0)
        at org.apache.lucene.util.hnsw.InitializedHnswGraphBuilder.addGraphNode(InitializedHnswGraphBuilder.java:489)
        at org.apache.lucene.util.hnsw.TestInitializedHnswGraphBuilderNullNodes.testAddNodeWithNullInitializedNodes(TestInitializedHnswGraphBuilderNullNodes.java:37)
1 test completed, 1 failed

# with -Ptests.asserts=false: the same exception and frames

Why it looks unintended

The public fromGraph javadoc allows null ("can be null if not tracking") and returns the builder "ready for additional operations"; the class advertises "incremental addition of new nodes".
HnswConcurrentMergeBuilder's private worker has the same override and treats null as "nothing to skip".
The exception is incidental: no explicit check, no Lucene error message, no test or discussion expecting it.

The full argument172 words

Contract. @param initializedNodes ... (can be null if not tracking) (InitializedHnswGraphBuilder.java:109-110) places no limit on later use; the factory returns the builder "ready for additional operations" (:99-101); the class javadoc lists "Allows incremental addition of new nodes while preserving initialized nodes" (:45); HnswBuilder.addGraphNode "Inserts a doc with a vector value to the graph" (HnswBuilder.java:39).
Sibling. The same override in HnswConcurrentMergeBuilder's private ConcurrentMergeWorker guards the same read (HnswConcurrentMergeBuilder.java:493, :561).
Tests. Every upstream fromGraph test passes a bit set (HnswGraphTestCase.java:701, :738); no test passes null or expects an exception.
History. The read dates from #12657 (ea272d0eda1 on main; shipped in 9.9.0 through its branch_9x copy 18bb826564b), when the @param read "a bitset of nodes that are already initialized in the initializerGraph", so null was caller misuse. #15003 (ef10476d98a, 10.4.0) added "(can be null if not tracking)" and the first null caller without guarding the read; its body, comments and review comments never discuss null tracking.

The devil's-advocate check160 words

The strongest case for "intended": the same commit wrote the initGraph javadoc, "create a fully initialized on-heap HNSW graph without tracking initialized nodes. This is useful when you just need the resulting graph structure without planning to add additional nodes incrementally" (InitializedHnswGraphBuilder.java:171-173). MergingHnswGraphBuilder uses null to mean "all nodes should be already initialized" and then inserts nothing (MergingHnswGraphBuilder.java:89-90, :167). And with the guard alone, build(maxOrd) on a builder created with null would quietly re-insert the copied nodes. So "not tracking" may have meant "copy only".

Why it fails: that sentence describes initGraph's use case, not a precondition of the public fromGraph, whose @param is what a caller reads; for a new node "not tracking" has one obvious meaning, which HnswConcurrentMergeBuilder's worker implements. It does make a reject-null fix as reasonable as the guard. The devil's-advocate verifier judged that it weakens but does not refute the claim; the cross-model reviewer rated it moderate.

Suggested fix

Recommended Skip the bit-set lookup when there is no bit set (fix A)

Guard the read as HnswConcurrentMergeBuilder does: if (initializedNodes != null && initializedNodes.get(node)) (one line). The repro then passes with assertions on and off, and all 139 tests of the HNSW package and TestHnswMergeAbort pass. Alternative: make null an explicit copy-only mode that rejects insertion.

  @Override
  public void addGraphNode(int node) throws IOException {
    if (initializedNodes != null && initializedNodes.get(node)) {
      return;
    }
    super.addGraphNode(node);
  }

All options and trade-offs2 options · 411 words

Fix A was built and checked through gradle on 5adff78bb168; B was not run.

Skip the bit-set lookup when there is no bit set (fix A) (validated by the checks) (recommended)

In InitializedHnswGraphBuilder.addGraphNode (:489), test for null first, exactly as HnswConcurrentMergeBuilder.java:561 does. Null then means "nothing to skip"; a sentence in the @param ("if null, no node is skipped") would make that explicit.

  @Override
  public void addGraphNode(int node) throws IOException {
    if (initializedNodes != null && initializedNodes.get(node)) {
      return;
    }
    super.addGraphNode(node);
  }

Trade-offs

With the guard alone, build(maxOrd) on a builder created with null no longer throws but re-inserts the copied nodes and replaces their copied level-0 neighbour arrays (the devil's-advocate verifier ran build(3) after copying a 2-node graph: 3 nodes, each linked to the other two at level 0, with node 0's copied neighbour array replaced, so the copy is wasted). If open PR #16815 lands first, its new fromGraph overload needs the same care.

What the checks validated

Gradle in a scratch worktree at 5adff78bb168, seed DEADBEEF: the repro fails without the fix (assertions on and off) and passes with it (assertions off).
With the fix and assertions on: every org.apache.lucene.util.hnsw test (11 classes plus the repro, including TestHnswFloatVectorGraph, TestHnswByteVectorGraph and TestHnswFloat16VectorGraph) and TestHnswMergeAbort: 139 tests, 0 failures, 3 skipped. checkGoogleJavaFormat passes. The full core suite was not run, and only OpenJDK 25.0.3 on Linux aarch64 was used.
Earlier: the hunt's shadowed runs of the same diff pass the repro (assertions on and off) and TestHnswFloatVectorGraph (32 tests); the cross-model reviewer's test and its bit-set control both pass with it.

Make null an explicit copy-only mode (not run)

Keep null for the internal copy-only paths (initGraph, pruneGraph), but throw IllegalStateException("builder was created without initializedNodes; it cannot add nodes") in addGraphNode when the bit set is null, or require a non-null bit set in the public fromGraph and give initGraph its own path; correct the @param to match.

Trade-offs

Matches the initGraph javadoc and MergingHnswGraphBuilder's reading of null, and keeps build() from silently re-inserting. It turns the documented call into a documented error instead of making it work, and is a slightly larger change. An independent fact check of the brief (OpenAI GPT-5.6) preferred this fix as the default, because it makes the copy-only reading explicit and does not silently change what build(maxOrd) does. The regression test would assert the IllegalStateException.

What the checks validated

Not run.

Downstream note

Downstream products are affected only where their own code builds graphs with fromGraph and a null bit set; documents, vectors and queries cannot make Lucene take this path.

How it was found

A proof-obligation audit of InitializedHnswGraphBuilder's contract (the fromGraph javadoc and addGraphNode): the null initializedNodes that the javadoc allows must leave the builder usable for addGraphNode. It does not, because addGraphNode reads it unguarded; a public-API JUnit test then witnessed the exception.

The method in full155 words

Method: semi-formal (a proof-obligation audit of a class's contract, witnessed by an executable repro).

The finder treated the fromGraph javadoc and the field and class javadoc as obligations on the returned builder: the null that the @param allows must be safe for a later addGraphNode. addGraphNode dereferences it with no check, while the same override in HnswConcurrentMergeBuilder's private ConcurrentMergeWorker guards it.
The finder then wrote a public-API JUnit test, ran it on main and on the 10.6.0 and 10.4.0 jars, and checked the one-line guard against it and TestHnswFloatVectorGraph. A supporting probe on a 50-node graph showed that with the guard, addGraphNode(50) on a builder created with null gives the same graph size (51) and the same level-0 degree for the new node (3) as the bit-set control.
An independent reproducer, an executed cross-model review, a devil's-advocate verifier and an adjudication reproduced it with their own tests ("Prior reports, history and checks").

Prior reports, history and checks

Prior reports

No matching prior report found in apache/lucene issues, PRs and commits, the mailing lists, downstream trackers or the web (searched 2026-11-03). Nearest: #15003 (the introducing PR, whose review never discusses null tracking) and #12657 (origin of the read).

Searches3 searches

No matching public report was found in the documented searches. Nearest items, none a report of this bug: #15003 (the introducing PR; its body, comments and review comments never discuss null tracking), #12657 (origin of the read), #16726 and #16729 (open; other parts of the same builders). Open PRs #16956, #16815 and #16555 touch the file but not the read or the @param; #16815 would add one more fromGraph overload with the same nullable parameter. JIRA cannot hold a report: the class was added in October 2023, after the move to GitHub issues. Limits: keyword search; GitHub search does not index PR diffs or every review comment; private reports are not visible.

What was searched, when, and what turned up
Where	When	How	Found
GitHub issues, PRs and commits (apache/lucene incl. the migrated JIRA archive)	2026-11-03; re-checked the same day	a scripted prior-report search (16 GitHub, 3 commit and 4 JIRA queries among 53) and manual searches for the class, the field, fromGraph null, addGraphNode NullPointerException and the exact JVM message; every PR updated since 2026-11-01	no report; #15003 and #12657 are history, not reports
Open PRs	2026-11-03	the changed files of all 409 open apache/lucene PRs	#16956, #16815, #16555 touch the file, not the read or the @param
Mailing lists, downstream, web	2026-11-03	dev@, java-user@, issues@ and commits@ (20 queries); Elasticsearch, OpenSearch, the OpenSearch k-NN plugin and Solr; 8 web queries	nothing about this bug

History

Introduced in 10.4.0 by #15003 (ef10476d98a, merged 2025-12-01), which documented null as allowed without guarding the older read from #12657 (9.9.0). 10.4.0, 10.5.2, 10.6.0 and main fail; not fixed upstream.

Versions tested6 rows

Introduced by #15003 "Avoid reconstructing HNSW graphs during segment merging." (Pulkit Gupta): ef10476d98a on main (merged 2025-12-01), cherry-picked to branch_10x as 3e024479d35; listed in 10.4.0's CHANGES (lucene/CHANGES.txt:1481). It added "(can be null if not tracking)" to the @param and made initGraph pass null. The unguarded read is older: #12657 (ea272d0eda1 on main, 2023-10-17; shipped in 9.9.0 through its branch_9x copy 18bb826564b), when the @param required a bit set.

releases/lucene/10.4.0 contains 3e024479d35 and 10.3.2 does not. 10.5.0 and 10.5.1 were not run; they contain the same @param and read. Upstream main 1c2072f0284 (2026-11-02), branch_11x and branch_10_6 still have both.

Versions and branches executed, with results
Version or commit	Verdict	Result
9.8.0	API absent	the class does not exist
9.9.0, 9.12.3, 10.3.2	API-adapted copies	the same exception, but the javadoc required a bit set there: caller misuse, not counted
10.4.0, 10.6.0	the repro, unchanged	fails, exception at :448 and :488 (-da; the hunt's 3-vector test with -ea too)
10.5.2	the reproducer's test	fails, exception at :464
main 5adff78bb168	JUnit, -ea and -da	fails, exception at :489
main 5adff78bb168 + fix A	gradle	passes; 139 HNSW tests pass

Independent checks

A finder, an independent reproducer, an executed cross-model review (OpenAI GPT-5.6, real, high), a devil's-advocate verifier (leaning real at Low), an adjudication, an audit and a final repro check reproduced it. This doc's gradle runs reproduced it on main and validated fix A.

Checks and verdicts224 words

The cross-model review ran on main ea57360912ed, whose InitializedHnswGraphBuilder.java is identical to 5adff78bb168's, and its test was re-run on 5adff78bb168 and 10.6.0. Every other check ran on 5adff78bb168 or release jars.

Each independent check and its verdict
Check	Verdict	What it found
finder	reproduced	Own test fails on main (assertions on and off), 10.6.0 and 10.4.0; the guard makes it pass, and `TestHnswFloatVectorGraph` passes with it.
independent reproducer	real	Own test, recorded as written from the claim only, fails on main (on, off), 10.6.0, 10.5.2 and 10.4.0; its bit-set control passes.
cross-model review, refute lens (OpenAI GPT-5.6, executed)	real (high)	Own 1-node test fails with assertions on and off and a second seed; the bit-set control passes; the guard passes both. Intended-behaviour argument rated moderate.
devil's-advocate verifier	uncertain, leaning real at Low	Built the `initGraph` case for "copy only"; judged it weakens but does not refute the claim. Ran `build()` with the guard: the copied nodes are re-inserted.
adjudication, separate re-runs, audit and final repro check	confirmed, Low	Fresh-seed re-runs on main fail and pass with the guard; one minimal deterministic test.
this doc's fix check	reproduced; fix A validated	Gradle without the fix fails (on, off); with it the repro and 139 HNSW tests pass. The repro also fails on the 10.6.0 and 10.4.0 jars.

Diagram file

The diagram as a still image, generated with AI and checked against the code. It is under 10 MB, the limit for a GitHub attachment.

PNG 474 KB

Sources and provenance

Finding record, Lucene and upstream sources, repro files and diagram notes1,537 words

Finding record

LFV-1197, snapshot of 2026-11-06

Registry title: InitializedHnswGraphBuilder.fromGraph documents initializedNodes as 'can be null if not tracking', but the returned builder's addGraphNode(int) reads it without a null check, so a builder created with null throws NullPointerException on its first insertion (since 10.4.0)
Area: core / util/hnsw: InitializedHnswGraphBuilder (fromGraph, addGraphNode; @lucene.experimental)
Priority: Low, rank 698: The experimental InitializedHnswGraphBuilder.fromGraph accepts a null initializedNodes as its javadoc allows, but the first insertion through the returned builder throws NullPointerException, in releases 10.4.0 through 10.6.0. Lucene's own merges never insert through it, so only custom graph-building code hits it, loudly and before the graph changes.
Impact class: minor
Impact note: Only code that calls the public @lucene.experimental InitializedHnswGraphBuilder.fromGraph itself with initializedNodes = null, as its javadoc allows, and then inserts a node through the returned builder (addGraphNode, or build) is hit: the first insertion throws NullPointerException with -ea and -da [exec], before the graph changes; Lucene's own merge paths (initGraph, pruneGraph) pass null but never insert through that builder, so IndexWriter, merges, search and every default configuration never reach it; 10.4.0 through 10.6.0 and main have it.
Summary: You copy an existing HNSW graph into a new builder with InitializedHnswGraphBuilder.fromGraph, passing null for initializedNodes, which its javadoc allows ('can be null if not tracking'), and then add a new vector with addGraphNode. You expect the vector to be inserted (with a bit set marking the copied nodes, the graph grows from 2 to 3 nodes). Instead addGraphNode throws NullPointerException ('Cannot invoke BitSet.get(int) because this.initializedNodes is null'), because it checks the bit set for nodes to skip without first checking that there is one.
Introduced: ef10476d98ab37b92fe48a1f71a1a06dad009ae3 (apache/lucene#15003 'Avoid reconstructing HNSW graphs during segment merging', by Pulkitg64; branch_10x cherry-pick 3e024479d358e77ba7d802308b70ccb25ccf08c4) [read: git log -S '(can be null if not' in the full history clone finds only these two commits, and blame of the @param line gives ef10476d98ab; the commit added '(can be null if not tracking)' to the fromGraph @param and made initGraph pass null, without guarding addGraphNode]. The unguarded read itself is older: 'if (initializedNodes.get(node))' dates from #12657 (ea272d0eda1 on main, 18bb826564b on branch_9x, by benwtrent, 2023-10-17, first released in 9.9.0) [read: git log -L], but through 10.3.2 the @param required a bit set and no in-tree code passed null, so a null there was caller misuse. (2025-12-01 (#15003 merged 2025-12-01T22:14:21Z; author date 2025-12-02T03:44:21+05:30; branch_10x cherry-pick committed 2025-12-01T17:49:47-05:00))
First affected version: 10.4.0 [exec: the unchanged repro fails there with the same NullPointerException at InitializedHnswGraphBuilder.java:448; its CHANGES lists GITHUB#15003]; 10.3.2, the last release before, is not affected as a contract bug [read: its @param reads 'a bitset of nodes that are already initialized in the initializerGraph', so null was off-contract; exec: an M-argument adaptation of the repro throws the same NullPointerException at :116, as caller misuse]
Affected versions: 10.4.0, 10.5.0, 10.5.1, 10.5.2 and 10.6.0 (released 2026-10-29, 4ae946e5a9b) and main 5adff78bb168 (12.0.0-SNAPSHOT); never fixed. [exec, JDK 25.0.3, Linux aarch64] finder: main (-ea, -da), 10.6.0 and 10.4.0 (-ea) fail with NullPointerException at InitializedHnswGraphBuilder.addGraphNode (:489 on main, :488 on 10.6.0, :448 on 10.4.0); history step: 10.4.0, 10.6.0 and main (-ea) fail; the independent reproduction's own test: main (-ea, -da), 10.6.0, 10.5.2 (:464) and 10.4.0 (-ea) fail; the cross-model reviewer's own test: main (-ea, -da) and 10.6.0 (-da) fail; the intended-behaviour check's own test: main (-ea, -da) and 10.6.0 (-da) fail; judge and re-runs on main (-ea, -da) fail. [read] 10.5.0 and 10.5.1 contain 3e024479d35 with the same @param and the same unguarded read. Before 10.4.0 (9.9.0 to 10.3.2) the same NullPointerException follows a null argument [exec: 9.9.0, 9.12.3, 10.3.2 with API-adapted copies], but the javadoc did not allow null there, so those releases are not counted. 9.8.0 and earlier have no such class. Not new in 10.6 (10.4.0 and 10.5.2 fail). Not fixed upstream: upstream main 1c2072f0284 (author date 2026-11-02), branch_11x and branch_10_6 still have the '(can be null if not tracking)' @param and the unguarded read (:489, :489, :488); the newest commit on the file is d8468fd9375 (#16627, author date 2026-10-08).
Status: documented (brief, checks and a per-finding doc exist)
Verification: Confirmed by AI-run checks: an independent repro, a cross-model review that agreed, an intended-behaviour check, a prior-report search and a skeptical audit; not yet triaged by a person.
Found by: semi-formal: proof-obligation audit
Upstream and prior reports: none found (2026-11-03, on top of the finder's, the intended-behaviour check's and the adjudication's searches; rechecked the same day): a follow-up scripted prior-report search (16 GitHub, 3 commit, 2 downstream, 20 mailing-list (dev@, java-user@, issues@, commits@), 4 JIRA and 8 web queries: 80 distinct hits, none high, 3 medium, none a report), the finder's run (45 queries, 59 distinct hits, none a report), manual GitHub issue, PR and commit searches in apache/lucene ('InitializedHnswGraphBuilder', 'initializedNodes', 'fromGraph null', 'addGraphNode NullPointerException', the NullPointerException message, 'InitializedHnswGraphBuilder javadoc', 'hnsw initialized nodes null') and in Elasticsearch, OpenSearch, the OpenSearch k-NN plugin and Solr, the changed files of 409 open apache/lucene PRs (#16956, #16815, #16555 touch the file but not the read or the @param), and the file history on main (author dates). The same-day recheck found 'fromGraph null', 'addGraphNode NullPointerException', '"this.initializedNodes" is null' and 'not tracking hnsw' returning nothing in apache/lucene, 'InitializedHnswGraphBuilder' and 'initializedNodes' returning only the already-classified history items, the only commit hit for initializedNodes being #12657, a GitHub-wide search for InitializedHnswGraphBuilder returning only those items plus two unrelated Elasticsearch issues (#158407, a merge-abort stall; #160380, a CI assertion) and one unrelated outside issue, no GitHub-wide hit for 'initializedNodes NullPointerException', only #16956 (open; its one changed line in the file removes scorer.setScoringOrdinal) among the 28 open PRs and only #16806 (merged, part of 5adff78bb168; HnswConcurrentMergeBuilder) among the 13 closed PRs updated since 2026-11-01 touching an HNSW builder file, and upstream main 1c2072f0284, branch_11x and branch_10_6 still unguarded. Related history, not reports: #15003 (the introducing PR; its body, 373 lines of comments and 32 review comments never discuss null tracking), #12657 (origin of the read), #16726 (open; rebalanceGraph seeding in the same class), #16729 (open; concurrent merge work partitioning), #16815 (open; would add one more fromGraph overload with the same nullable parameter).
Lucene commit tested: main 5adff78bb168 (2026-11-02) through gradle on 2026-11-03 in a scratch worktree, seed DEADBEEF: the repro without the fix (assertions on and off: fails) and with fix A (assertions off: passes; assertions on, with every org.apache.lucene.util.hnsw test and TestHnswMergeAbort: 139 tests, 0 failures, 3 skipped), plus checkGoogleJavaFormat; the same repro on the 10.6.0 and 10.4.0 jars (assertions off: fails). The hunt ran the repro or an equivalent JUnit test on main (assertions on and off), 10.6.0, 10.5.2 and 10.4.0 (fail), and API-adapted copies on 9.9.0, 9.12.3 and 10.3.2 (the same exception, but there the javadoc required a bit set). Upstream main 1c2072f0284, branch_11x and branch_10_6 were read on 2026-11-03. Only Linux aarch64 with OpenJDK 25.0.3 was run.

Lucene and upstream sources

Finding record: lucene/core/src/java/org/apache/lucene/util/hnsw/InitializedHnswGraphBuilder.java:33-56 (at 5adff78bb168) (class javadoc; :45 'Allows incremental addition of new nodes while preserving initialized nodes'; :54 @lucene.experimental; :56 public final class), :58-62 (initializedNodes field javadoc: 'These nodes will be skipped during subsequent addGraphNode(int) calls'), :96-119 (fromGraph javadoc; :100 'returns the builder ready for additional operations'; :109-110 '@param initializedNodes bit set marking which nodes are already initialized (can be null if not tracking)'), :120 and :150 (the two public fromGraph overloads, argument stored unchecked), :170-206 (initGraph; :171-173 'without tracking initialized nodes ... without planning to add additional nodes incrementally'; null passed, builder discarded), :224-234 (private constructor, :232 this.initializedNodes = initializedNodes), :249-269 (package-private pruneGraph, passes null, no insertion), :487-493 (addGraphNode; :489 'if (initializedNodes.get(node))': fix site)
lucene/core/src/java/org/apache/lucene/util/hnsw/HnswConcurrentMergeBuilder.java:559-565 (at 5adff78bb168) (worker addGraphNode: 'if (initializedNodes != null && initializedNodes.get(node))', the guarded sibling), :119-121 and :145-148 (its own builder requires a non-null set when graphs is not empty); lucene/core/src/java/org/apache/lucene/util/hnsw/MergingHnswGraphBuilder.java:89-90 ('if null, all nodes should be already initialized after updateGraph') and :167 (inserts nothing when null); lucene/core/src/java/org/apache/lucene/util/hnsw/ConcurrentHnswMerger.java:78-83 (own FixedBitSet for insertion, pruneGraph for the base graph); lucene/core/src/java/org/apache/lucene/util/hnsw/HnswBuilder.java:32-40 (build(maxOrd) 'Adds all nodes'; addGraphNode 'Inserts a doc with a vector value to the graph')
lucene/CHANGES.txt:1481 (at 5adff78bb168) (10.4.0: 'GITHUB#15003: Avoid reconstructing HNSW graph during merging') and :1432 ('GITHUB#15184: Refactoring internal HNSWGraphBuilder's APIs ...')
The finding in 30 seconds: lucene/core/src/java/org/apache/lucene/util/hnsw/InitializedHnswGraphBuilder.java:45, (at 5adff78bb168) :54-56, :58-62, :96-119, :120-168, :170-207, :224-234, :249-270, :487-493
lucene/core/src/java/org/apache/lucene/util/hnsw/HnswConcurrentMergeBuilder.java:559-565; (at 5adff78bb168) HnswGraphBuilder.java:201-208; ConcurrentHnswMerger.java:75-83
Who is affected: lucene/core/src/java/org/apache/lucene/util/hnsw/InitializedHnswGraphBuilder.java:54-56, (at 5adff78bb168) :187-207, :249-270; ConcurrentHnswMerger.java:78-83; MergingHnswGraphBuilder.java:108
lucene/core/src/java/module-info.java:50 (at 5adff78bb168)
lucene/core/src/test/org/apache/lucene/util/hnsw/HnswGraphTestCase.java:700-715, (at 5adff78bb168) :737-754
lucene/core/src/java/org/apache/lucene/util/hnsw/InitializedHnswGraphBuilder.java:487-493 (at 5adff78bb168) (the read is the first statement of addGraphNode(int))
Why it looks unintended: lucene/core/src/java/org/apache/lucene/util/hnsw/InitializedHnswGraphBuilder.java:45, (at 5adff78bb168) :58-62, :99-101, :109-110, :171-173, :487-493; HnswBuilder.java:32-40; HnswConcurrentMergeBuilder.java:493, :559-565; MergingHnswGraphBuilder.java:89-90, :167
lucene/core/src/test/org/apache/lucene/util/hnsw/HnswGraphTestCase.java:701, (at 5adff78bb168) :738 (grep of lucene/*/src/test: no fromGraph call with null)
Suggested fix: lucene/core/src/java/org/apache/lucene/util/hnsw/HnswConcurrentMergeBuilder.java:559-565; (at 5adff78bb168) InitializedHnswGraphBuilder.java:170-207, :249-270, :487-493; MergingHnswGraphBuilder.java:89-90, :167
Prior reports: https://github.com/apache/lucene/pull/15003, https://github.com/apache/lucene/pull/12657, https://github.com/apache/lucene/pull/16815
History: lucene/CHANGES.txt:1481 (at 5adff78bb168)
Diagrams: lucene/core/src/java/org/apache/lucene/util/hnsw/InitializedHnswGraphBuilder.java:109-110, (at 5adff78bb168) :487-493

Other statements trace to the hunt's working files (models, logs, reviews and briefs), which are not published.

Repro files

TestInitializedHnswGraphBuilderNullNodes.java (lucene/core, package org.apache.lucene.util.hnsw, 1 test): repro/LFV-1197/TestInitializedHnswGraphBuilderNullNodes.java (40 lines)

About the diagrams

The diagram follows the repro's inputs and the executed outcomes: the exception on main and the bit-set control that gives a 2-node graph (InitializedHnswGraphBuilder.java:109-110, :487-493).

assets/LFV-1197/bug.png (474 KB)

Built 2026-11-06 from the hunt's record of this finding and its doc file. Lucene main under test in this hunt: ea57360912ed. Every listed finding was re-run on Lucene main e1faeba08bde on 2026-11-02.