Eight analysis-factory javadoc schema examples are malformed XML or name a class that is not a factory

Most Lucene analysis factories show, in their class javadoc, the XML you would put in a Solr-style schema to use them, for example <filter class="solr.StopFilterFactory" words="stopwords.txt"/>. In eight factories that snippet is broken. Six are not well-formed XML: StopFilterFactory (the <filter start tag is never closed), JapaneseIterationMarkCharFilterFactory (the class value has no closing quote after the class name, so the next quote ends it), BeiderMorseFilterFactory (no > before </filter>), OpenNLPTokenizerFactory and OpenNLPLemmatizerFilterFactory (the <fieldType start tag has no >, plus a stray /> line in the second) and DropIfFlaggedFilterFactory (<-- other filters --> is not an XML comment). Two parse but name a class that is not a factory: LimitTokenOffsetFilterFactory says solr.LimitTokenOffsetFilter, and PatternTypingFilterFactory's own element is com.example.PatternTypingFilter; its next line names solr.TokenAnalyzerFilter, a filter from a Solr patch that was never committed, and its first sentence describes that filter. Copying the StopFilterFactory snippet gives Element type "filter" must be followed by either attribute specifications, ">" or "/>" at line 6, column 3. No Lucene code parses these snippets, and every factory argument in them (each attribute other than class, the TokenAnalyzerFilter line aside) is one its factory reads, so only readers who copy them are hit. All eight are in every release since 8.9.0, the oldest two since 4.0.0-BETA. A javadoc-only fix (8 files, 11 insertions, 13 deletions) makes all eight parse and name registered factories.

The control, LimitTokenPositionFilterFactory lines 26-32, has the same shape and is correct.

Checked against Lucene's own SPI registrations (the META-INF/services/org.apache.lucene.analysis.*Factory files of lucene-core and the analysis modules: 9 char-filter, 19 tokenizer and 138 token-filter factories): LimitTokenOffsetFilter, PatternTypingFilter and TokenAnalyzerFilter are not among them.

No test or build task reads language-xml blocks, and javadoc rendering accepts the broken text. #15573 (2026-01-15) rewrote every one of these blocks from <pre class="prettyprint"> to <pre><code class="language-xml"> and kept all eight defects.

Any runtime effect: every factory argument in the eight snippets (each attribute other than class) is one its factory reads, the never-committed TokenAnalyzerFilter line aside. The exact error a particular schema host prints: no real schema loader was run. The two synonym-factory examples' [optional tokenizer factory parameters] placeholder is intentional and not reported.

• Who. Readers who copy a snippet into an XML schema of a Solr-style host. No query, document or index file reaches the text.
• Default vs opt-in. StopFilterFactory is mainstream; LimitTokenOffset, PatternTyping and DropIfFlagged are rarely used filters; the other four live in optional modules.
• Scale. No size matters: the copied snippet fails every time. Every release from 4.0.0-BETA through 10.6.0 (132 releases) has at least two of the eight; all eight since 8.9.0.
• Effect. Loud, not silent: a strict XML parser rejects the six malformed snippets at once, and nothing in Lucene is indexed or searched differently (lenient parsers and real schema loaders were not tested).
• Priority. Low (rare, documentation only), but cheap to fix and easy to review.

The text block is the javadoc block StopFilterFactory.java:29-36 at 645d4fe14a20 with the * prefix removed and the entities unescaped; a script found the two identical there and at 10.5.2, and the same extraction on the unpatched tree regenerates the test's text exactly. Because the test embeds the snippet, on a release jar it re-tests the text; release dating rests on reading each release's sources.

Inside a Lucene checkout with gradle: put the file in lucene/analysis/common/src/test/org/apache/lucene/analysis/core/ and run ./gradlew :lucene:analysis:common:test --tests org.apache.lucene.analysis.core.TestStopFilterFactoryJavadocExample. It fails there too (1 test, 1 failure, the same message), and its fixed twin passes.

All eight rows, read from source (Python stdlib XML parser plus a lookup of each solr.<Name> in the checkout's SPI service files; main 645d4fe14a20):

StopFilterFactory.java:29-36                      not well-formed XML (line 7, column 2)
JapaneseIterationMarkCharFilterFactory.java:26-32 not well-formed XML (line 4, column 83)
BeiderMorseFilterFactory.java:31-40               not well-formed XML (line 8, column 4)
OpenNLPTokenizerFactory.java:33-38                not well-formed XML (line 3, column 2)
OpenNLPLemmatizerFilterFactory.java:32-43         not well-formed XML (line 3, column 2)
DropIfFlaggedFilterFactory.java:32-38             not well-formed XML (line 4, column 7)
LimitTokenOffsetFilterFactory.java:26-32          <filter class="solr.LimitTokenOffsetFilter"> names no registered TokenFilterFactory
PatternTypingFilterFactory.java:36-45             com.example.PatternTypingFilter and solr.TokenAnalyzerFilter name no registered TokenFilterFactory
LimitTokenPositionFilterFactory.java:26-32        OK (control)

(This checker keeps the empty line after <pre><code>, so its line numbers are one higher than the JDK parser's.)

No separate regression test is suggested: this test copies the example, so it would not notice a later edit of the javadoc; the real guard is a build check over all language-xml blocks (Suggested fix, option C).

# the command above, main 645d4fe14a20, seed DEADBEEF, -ea (-da identical); banner and NOTE lines left out, ... = remaining runner frames
[Fatal Error] :6:3: Element type "filter" must be followed by either attribute specifications, ">" or "/>".
There was 1 failure:
1) testExampleIsWellFormedXml(org.apache.lucene.analysis.core.TestStopFilterFactoryJavadocExample)
java.lang.AssertionError: expected well-formed XML, but: Element type "filter" must be followed by either attribute specifications, ">" or "/>".
	at __randomizedtesting.SeedInfo.seed([DEADBEEF:3AF316B5A8EAB8DD]:0)
	at org.apache.lucene.analysis.core.TestStopFilterFactoryJavadocExample.testExampleIsWellFormedXml(TestStopFilterFactoryJavadocExample.java:45)
	...
Caused by: org.xml.sax.SAXParseException; lineNumber: 6; columnNumber: 3; Element type "filter" must be followed by either attribute specifications, ">" or "/>".
	...
FAILURES!!!
Tests run: 1,  Failures: 1

# standalone on main (-ea, -da) and on the 10.5.2 release jars (-ea, -da): the same failure

• Offered as configuration. StopFilterFactory.java:27-29 reads "Factory for {@link StopFilter}." and then the block, with no "schematic" marker; DropIfFlaggedFilterFactory.java:30 says "In Solr this might be configured such as". No package doc, CHANGES or MIGRATE entry calls the snippets schematic.
• Maintained as live XML. #15573 (Dawid Weiss, 2026-01-15) marked every block language-xml. New factories still get one: #16371 CaseFoldingFilterFactory (2026-07-31); #16722 extended the ThaiTokenizerFactory example (2026-09-28).
• The convention holds elsewhere. 141 of the 149 example blocks in all factory javadocs are well-formed (6 of the other 8 are this bug, 2 are the synonym factories' intentional placeholder), and 328 of their 332 elements name a registered factory (the 4 that do not are this bug).
• Broken by an unrelated edit. In 4.x the StopFilterFactory example ended words="stopwords.txt" enablePositionIncrements="true"/>; LUCENE-4963 (Adrien Grand, 2013-05-04, "Completely remove deprecated options in 5.0.") removed the attribute and the /> with it.
• Precedent. LUCENE-8523 (Alan Woodward, 2018-10-14) changed solr.JapaneseNumberFilter to solr.JapaneseNumberFilterFactory, with the CHANGES entry "Correct typo in JapaneseNumberFilterFactory javadocs"; SOLR-6000 (2014-04-21) fixed the same in PatternCaptureGroupFilterFactory, solr.PatternCaptureGroupTokenFilter to solr.PatternCaptureGroupFilterFactory, in two commits.

Strongest case for "not a bug" (the intended-lens and refute reviewers, OpenAI GPT-5.6, and the devil's-advocate verifier): the snippets are Solr-specific illustrations, not input to any Lucene API (AbstractAnalysisFactory documents a constructor / inform / create() lifecycle, and Lucene parses no schema XML); DropIfFlaggedFilterFactory only says Solr "might" be configured so; com.example reads as a placeholder; and open issues #11285, #9013 and #9916 want these snippets removed altogether.

Why it fails: that argues for deleting the snippets, not for keeping broken ones. While they exist, upstream maintains them as XML and has fixed this exact class-name slip before (SOLR-6000, LUCENE-8523). A missing quote or > is not a placeholder. The case holds only for the TokenAnalyzerFilter line (deliberate, now stale) and, weakly, for the <-- other filters --> line. Every verifier and the adjudication called it real; the adjudication rated it a minor documentation defect.

One fix was validated, with the javadoc diff applied to a scratch worktree of 645d4fe14a20; the other options were not run. A maintainer may prefer option B (open issue #11285); option A does not block it.

Javadoc only. Lines are at 645d4fe14a20; -/+ show the javadoc text as it is in the files.

StopFilterFactory.java:34
- *             words="stopwords.txt" format="wordset"
+ *             words="stopwords.txt" format="wordset"/>
JapaneseIterationMarkCharFilterFactory.java:29
- *     <charFilter class="solr.JapaneseIterationMarkCharFilterFactory normalizeKanji="true" normalizeKana="true"/>
+ *     <charFilter class="solr.JapaneseIterationMarkCharFilterFactory" normalizeKanji="true" normalizeKana="true"/>
BeiderMorseFilterFactory.java:37
- *        concat="true" languageSet="auto"
+ *        concat="true" languageSet="auto">
OpenNLPTokenizerFactory.java:34 (and OpenNLPLemmatizerFilterFactory.java:33, name="text_opennlp_lemma")
- * <fieldType name="text_opennlp" class="solr.TextField" positionIncrementGap="100"
+ * <fieldType name="text_opennlp" class="solr.TextField" positionIncrementGap="100">
OpenNLPLemmatizerFilterFactory.java:38
- *     />
DropIfFlaggedFilterFactory.java:35
- *       <-- other filters -->
+ *       <!-- other filters -->
LimitTokenOffsetFilterFactory.java:30
- *     <filter class="solr.LimitTokenOffsetFilter" maxStartOffset="100000" consumeAllTokens="false" />
+ *     <filter class="solr.LimitTokenOffsetFilterFactory" maxStartOffset="100000" consumeAllTokens="false" />
PatternTypingFilterFactory.java:32-34
- * Provides a filter that will analyze tokens with the analyzer from an arbitrary field type. By
- * itself this filter is not very useful. Normally it is combined with a filter that reacts to types
- * or flags.
+ * Factory for {@link PatternTypingFilter}, which sets the type and flags of tokens that match one
+ * of a set of regex patterns. By itself this filter is not very useful. Normally it is combined
+ * with a filter that reacts to types or flags.
PatternTypingFilterFactory.java:40-41
- *     <filter class="com.example.PatternTypingFilter" patternFile="patterns.txt"/>
- *     <filter class="solr.TokenAnalyzerFilter" asType="text_en" preserveType="true"/>
+ *     <filter class="solr.PatternTypingFilterFactory" patternFile="patterns.txt"/>

Trade-offs

None for users: documentation only. Dropping the TokenAnalyzerFilter line leaves a chain that exists in Lucene (PatternTypingFilter then TypeAsSynonymFilter, whose prefix and ignore arguments the example already uses); a placeholder comment there would also do. The new first sentence follows PatternTypingFilter's own javadoc (it sets a type and flags on matching tokens).

What the checks validated

• All nine checked blocks pass. The source check on the patched tree: every block well-formed and every <charFilter>, <tokenizer> and <filter> names a registered factory, 0 of 9 broken (8 of 9 on the stock tree). A Lucene-side check (XML parse plus SPI lookup), not a run of a real schema loader.
• The minimal test passes on the patched text. Its fixed twin, generated mechanically from the patched StopFilterFactory.java, passes 1 of 1 with the plain command (assertions on and off) and standalone (on and off); the stock test fails the same runs.
• The modules' tests and checks. ./gradlew with the eight factories' own test classes in :lucene:analysis:common, :kuromoji, :phonetic and :opennlp (-Ptests.seed=DEADBEEF): 8 suites, 33 tests, 0 failures. checkGoogleJavaFormat and renderJavadoc pass on the four modules, and on the rendered pages all nine examples (the eight and the control) parse as XML. A javadoc edit cannot change behaviour; these runs show it builds and breaks nothing.
• Independent corroboration. The 2026-11-01 refute reviewer (OpenAI GPT-5.6) wrote its own correction test with the same changes: 2 run, 0 failed on 645d4fe14a20.

Replace each Solr XML snippet by a list of parameters, as open issue #11285 (LUCENE-10249) proposes, or remove them, as #9013 (LUCENE-7964) and #9916 (LUCENE-8873) propose.

Trade-offs

A much larger change over about 146 factory files that the project has left open for years. Option A fixes today's text without blocking it.

A small test that extracts every <pre><code class="language-xml"> block from the factory sources, parses it and resolves each solr.<Name> through the SPI would stop this drift; this doc's source check is 84 lines of Python, the hunt's extractor 59.

Trade-offs

Build infrastructure for documentation, and it would encode Solr's naming convention in Lucene's build. Not part of the fix.

• Sweep. A hunt unit on "analysis-factory javadoc examples and resource files build as documented" extracted every <pre> schema example from the class javadoc of every *Factory.java in all non-sandbox modules, checked each block for XML well-formedness, resolved each element through the SPI under the solr.<Name> convention, built it with exactly the documented attributes, gave it a minimal valid resource of the documented format and analyzed a sample (2026-10-28).
• Controls. A known broken example (LFV-702, another factory) failed as expected, so the sweep is not vacuous; LimitTokenPositionFilterFactory's example is the per-test control.
• Tier. Semi-formal: an exhaustive check of every example against two exact rules (XML grammar, SPI registration), then a public-API JUnit.