The util.fst package overview's code examples call removed APIs and have not compiled on any release since 9.0.0

The org.apache.lucene.util.fst package overview (package-info.java) is the package's only end-to-end usage example: five code blocks that share one FST built from cat=5, dog=7, dogs=12 (build it, look up a key, look up by value, iterate in sorted order, N shortest paths), with the printed values in comments. Copied in page order with only imports added, they fail to compile on main with 6 javac errors in 3 of the 5 blocks: the public FSTCompiler(INPUT_TYPE, Outputs) constructor was removed in 9.9.0 (#12715; FSTCompiler.Builder is the API), Util.getByOutput was removed in 9.0.0 (LUCENE-8638, #249, which deleted the page's link to it but kept the code block), and Util.Result became a record in 10.0.0 (#13328), so .input / .output are private and need input() / output(). No release since 9.0.0 ships a compiling version of the page (1 error in 9.0.0-9.8.0, 2 in 9.9.0-9.12.3, 6 in 10.0.0-10.6.0 and main). It is a documentation defect only: nothing wrong ever runs. With a javadoc-only fix (three corrections: Builder, accessors, drop the lookup-by-value block, which has no named replacement; plus a one-word comment update) the page's blocks compile and print the values in their comments, with the iteration's sorted pairs in between.

The page's code sits inside <pre><code> in a javadoc comment. The javadoc build checks {@link} targets but never compiles code inside <pre>, so three API changes that each updated their callers in code left the page behind:

666c7a25907 also deleted the overview's bullet {@link org.apache.lucene.util.fst.Util#getByOutput Lookup-by-output} when the outputs are in sorted order (e.g., ordinals or file pointers) (a dangling link fails the javadoc build) and kept the code block that calls it. #12831 (2024) later edited line 52 of the construction block for the new compile() API and left line 45.

The page's own blocks, extracted mechanically from package-info.java at 645d4fe14a20 (only imports added), compiled against the core jar of the same commit (condensed: javac's messages verbatim, file paths, source and caret lines, the reason: and type-variable lines dropped, lines mapped to page lines):

error: constructor FSTCompiler in class FSTCompiler<T> cannot be applied to given types;   (line 45)
  required: INPUT_TYPE,double,Outputs<Long>,boolean,DataOutput,float,int
  found:    INPUT_TYPE,PositiveIntOutputs
error: cannot find symbol  symbol: method getByOutput(FST<Long>,int)  location: class Util   (line 66)
error: input has private access in Result    (line 92)
error: output has private access in Result   (line 93)
error: input has private access in Result    (line 94)
error: output has private access in Result   (line 95)
6 errors

The other two blocks, "Retrieval by key" (:57-60) and "Iterate over key-value pairs in sorted order" (:72-80), compile and print 7 and the sorted pairs. The page's warning that keys must be added in Unicode code point order, not String.compareTo order (:38-40), is correct and unaffected.

• Who. Users of FSTCompiler, FST, Util and BytesRefFSTEnum who build a custom dictionary, suggester or term index and learn the API from the package overview. The published javadoc shows the same page: an independent verifier found the stale lines on the published 10.3.1 and 9.12.0 package-summary pages.
• Default vs opt-in. No configuration is involved; anyone who copies the examples hits it, every time. The classes are @lucene.experimental, an expert API, which is why the impact class is minor.
• What they lose. Time: they have to find FSTCompiler.Builder and the record accessors on their own (CHANGES and MIGRATE name them, the page does not). And a wrong belief: "Retrieval by value" advertises lookup by output, which the public API no longer offers.
• Scale. 3 of 5 examples, 6 errors on main and all of 10.x; the page has not compiled as shipped in any release since 9.0.0 (2021).
• Not affected. Lucene's behaviour, results and indexes. javac names the missing members at once.

The test holds the code lines of the construction, retrieval-by-value and N-shortest-paths blocks verbatim and in page order from package-info.java (HTML entities decoded; the four comment-only lines 38-40 and 65 left out; 24 of the blocks' 28 non-blank lines, compared mechanically), adds only imports and an enclosing method, compiles them with javax.tools against the test's own classpath, and asserts that javac succeeds. No randomness, single thread; the seed does not matter. The two blocks that compile (retrieval by key, iteration) are left out to keep it minimal.

Under gradle, as an ordinary core test, it fails the same way: the assertion message is javac's diagnostic list, starting with the constructor error.

Regression test (optional): this same test with the three blocks as the fix rewrites them. It passes after the fix and fails again if FSTCompiler.Builder or the Util.Result accessors change shape; it cannot see later edits of the page itself, since the page is not on the test classpath. For a javadoc-only change a committer may not want a test at all. ./gradlew tidy (google-java-format) re-wraps its long comment line and the List.of(...) options; the test passes the lint and forbidden-APIs checks.

java.lang.AssertionError: [/Example.java:12: error: constructor FSTCompiler in class org.apache.lucene.util.fst.FSTCompiler<T> cannot be applied to given types;
FSTCompiler<Long> fstCompiler = new FSTCompiler<Long>(INPUT_TYPE.BYTE1, outputs);
                                ^
  required: org.apache.lucene.util.fst.FST.INPUT_TYPE,double,org.apache.lucene.util.fst.Outputs<java.lang.Long>,boolean,org.apache.lucene.store.DataOutput,float,int
  found:    org.apache.lucene.util.fst.FST.INPUT_TYPE,org.apache.lucene.util.fst.PositiveIntOutputs
  reason: actual and formal argument lists differ in length, /Example.java:21: error: cannot find symbol
IntsRef key = Util.getByOutput(fst, 12);
                  ^
  symbol:   method getByOutput(org.apache.lucene.util.fst.FST<java.lang.Long>,int)
  location: class org.apache.lucene.util.fst.Util, /Example.java:31: error: input has private access in org.apache.lucene.util.fst.Util.Result
System.out.println(Util.toBytesRef(paths.topN.get(0).input, scratchBytes).utf8ToString()); // cat
                                                    ^, /Example.java:32: error: output has private access in org.apache.lucene.util.fst.Util.Result
System.out.println(paths.topN.get(0).output); // 5
                                    ^, /Example.java:33: error: input has private access in org.apache.lucene.util.fst.Util.Result
System.out.println(Util.toBytesRef(paths.topN.get(1).input, scratchBytes).utf8ToString()); // dog
                                                    ^, /Example.java:34: error: output has private access in org.apache.lucene.util.fst.Util.Result
System.out.println(paths.topN.get(1).output); // 7
                                    ^]
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at org.apache.lucene.util.fst.TestFstPackageExamples.testPackageOverviewExamplesCompile(TestFstPackageExamples.java:87)

tests_run=1 failures=1 (main 645d4fe14a20, seed DEADBEEF, with -ea and with -da)

• The contract is the page. Headings "FST Construction example", "Retrieval by key", "Retrieval by value", "Iterate over key-value pairs in sorted order", "N-shortest paths by weight", and the printed values in // x comments (package-info.java:35-96). Every identifier the blocks use is declared on the page.
• Upstream keeps this page compiling. LUCENE-6112 (#7174, 2014, "Compile error with FST package example code", fixed as "Fix compile error in FST package example code"); the LUCENE-9089 follow-up 59a8e83520e ("update FST usage example", Tomoko Uchida, 2020-04-29), which rewrote the blocks for the then-new FSTCompiler and TopResults; #12831 (2024), which updated line 52 for the compile() change.
• Oversight. None of #249, #12715, #12831 or #13328 discusses the page's code (threads read). CHANGES and MIGRATE record the removals as deliberate API changes (naming the replacements for the constructor and the record fields), so the page documents APIs the project itself retired.
• Boundary isolated by execution. Each commit's own page was compiled against core built at that commit. The code blocks are the same at both commits (the 2020 text, which differs from today's at line 52, changed by #12831 in 2024, and in markup). They compile and print the commented values and the sorted pairs at a37844aedd5 (the parent of 666c7a25907) and fail with the getByOutput error at 666c7a25907.

Strongest case for "not a bug" (built by the intended-behaviour checkers): (a) the FST API is @lucene.experimental; on the #12715 review Mike McCandless wrote that the FST API is internal, so the old constructor could be removed without deprecation; (b) Lucene does not compile javadoc code: LUCENE-8488 (#9534, "Javadoc examples should be kept in sync with actual compiled code") is still open, and on #15225 Robert Muir wrote that code examples should be kept "extremely minimal" because "the maintenance cost is very high and we don't have great validation for these"; (c) the failure is loud, and CHANGES and MIGRATE name the replacements for the constructor and the record fields; (d) the blocks are fragments.

Why it fails: experimental status explains why the API could change, not why the package's own overview keeps showing the removed forms; the lack of validation explains how the drift got in, and nobody upstream has said stale examples are acceptable (LUCENE-8488 and #15225 point the other way); a loud error lowers the impact, but the lookup-by-value block still advertises a removed capability; and the fragments need only imports. Verdict: not intended, a documentation defect (Low). To concede: the example itself uses two @lucene.internal helpers (BytesRefBuilder, IntsRefBuilder), so this is a trivial documentation cleanup that maintainers may reasonably resolve by trimming or deleting the examples instead, in line with the views above.

Option A was validated in a scratch worktree of 645d4fe14a20; the others were not run.

In package-info.java: use FSTCompiler.Builder in the construction block, the record accessors in the N-shortest-paths block, remove the "Retrieval by value" block, and change "provided to Builder" to "provided to FSTCompiler" in the construction comment. 1 file, +6/-14, javadoc only.

- *     FSTCompiler<Long> fstCompiler = new FSTCompiler<Long>(INPUT_TYPE.BYTE1, outputs);
+ *     FSTCompiler<Long> fstCompiler = new FSTCompiler.Builder<Long>(INPUT_TYPE.BYTE1, outputs).build();
  ...
- * Retrieval by value:
- *     IntsRef key = Util.getByOutput(fst, 12);   (whole block removed)
  ...
- *     System.out.println(Util.toBytesRef(paths.topN.get(0).input, scratchBytes).utf8ToString()); // cat
- *     System.out.println(paths.topN.get(0).output); // 5
+ *     System.out.println(Util.toBytesRef(paths.topN.get(0).input(), scratchBytes).utf8ToString()); // cat
+ *     System.out.println(paths.topN.get(0).output()); // 5
  (the same for get(1))

Trade-offs

Smallest change; keeps the package's only end-to-end example and stops advertising a removed feature. It does not prevent future drift, since nothing compiles the page.

What the checks validated

Yes, in a scratch worktree of 645d4fe14a20. The diff applies cleanly. The patched page's own examples, extracted mechanically from package-info.java (only imports added), compile and print 7 cat 5 dog 7 dogs 12 cat 5 dog 7, the values in the page's comments plus the iteration's sorted pairs; the stock page gives the 6 errors. With gradle and the fix applied, the 9 org.apache.lucene.util.fst suites (38 tests, including the minimal test with the corrected blocks) pass with assertions on and off; renderJavadoc, checkGoogleJavaFormat, ecjLintMain and forbiddenApisMain of core pass. On the stock tree the minimal test fails with the 6 errors. The hunt's compile-and-run check of the patched page also passes, and a mutant with a wrong documented order fails, so the output comparison is not vacuous.

As A, but replace the "Retrieval by value" block with a sentence such as "Lookup by output was removed in 9.0 (LUCENE-8638)."

Trade-offs

Tells readers of older articles why the feature is gone; adds a history remark to a usage page, which Lucene usually leaves to CHANGES and MIGRATE.

What the checks validated

Not run (text only).

Move the examples into a test source and reference them from the javadoc (for example with {@snippet file=...}), or keep a test that mirrors them (the direction of LUCENE-8488).

Trade-offs

Prevents the next drift, but needs new build plumbing (main uses no {@snippet} today) and carries the maintenance cost maintainers warned about on #15225; a bigger discussion than this bug needs.

What the checks validated

Not run.

Keep only a pointer to FSTCompiler.Builder, Util and the enums, or remove the code blocks.

Trade-offs

No drift possible, but the package loses its only end-to-end usage example. Acceptable if a committer prefers it.

What the checks validated

Not run.

• Lead. A first pass that checks javadoc examples compiled the five blocks verbatim against the stock main jars and got all six errors (2026-10-29).
• Hunt. The hunt extracted the blocks mechanically (HTML entities decoded, only imports added), compiled them with javac against the core jar of every release 8.0.0-10.5.2, a 10.6 build of 4ae946e5a9b (then RC1, now the 10.6.0 tag), branch_10x and main (both with each release's own page and with main's page), scanned the three APIs with javap, and wrote a compile-and-run JUnit test that compares the printed lines with the documented values; a mutant with a wrong documented order fails, so the comparison is not vacuous. The one-test repro used here keeps only the compile step.
• Tier. Non-formal: compiling documentation code and scanning jars for API members.