jones^2 reaches the node unchanged: the grammar stores Float.parseFloat("2") (StandardSyntaxParser.jj:701), the constructor checks nothing (BoostQueryNode.java:47), and the builder makes BoostQuery 2.0. The javadoc's 0.0-1.0 range (BoostQueryNode.java:29, :39, :67) matches none of this.
StandardQueryParserworks in three phases: a syntax parser turns the text into a tree of query nodes, processors rewrite the tree, and builders turn each node into a LuceneQuery. Developers can plug in their own processors and builders.- The
^operator, as injones^2, becomes aBoostQueryNodearound the boosted part of the tree; itsgetValue()is the number after^. - A boost above 1 makes a clause count more in the score and a boost below 1 less, as
BoostQueryand the parser's syntax docs say. Boosts can also come from configuration:setFieldsBoostgives each field a boost.
jones^2 goes through the parser unchanged: the node holds 2.0, outside the 0.0-1.0 range its javadoc states, and BoostQuery 2.0 is built and scored correctly. Select the still to open it at full size. Generated with AI, checked against the code.- You write a custom processor or builder for the flexible query parser that reads
BoostQueryNode.getValue(). Its javadoc says "The boost value may vary from 0.0 to 1.0", and says it again on the constructor and ongetValue. - You parse
jones^2 OR smith^0.5, the example inStandardQueryParser's own javadoc, and expect node values between 0 and 1. - You get 2.0 and 0.5. Field boosts set with
setFieldsBoost, say 5 and 10, arrive as 5.0 and 10.0. Nothing clamps or checks the value, and nothing should. - Lucene itself is fine: it builds
BoostQuery2.0 and scores correctly. Only code that trusts the javadoc goes wrong, for example by clampingjones^2to 1.0 or rejecting it as out of range.
Developers who extend the flexible query parser with processors, builders or validators that read BoostQueryNode and trust the stated range. Users who only call StandardQueryParser.parse are not affected: queries are built and scored correctly. No setting or scale is involved; the text is in every release since 2.9.0.
Mechanism, terms and numbers440 words
In the flexible query parser (StandardQueryParser), a boost such as jones^2 becomes a BoostQueryNode in the query-node tree, and a builder later turns it into a BoostQuery. The node's javadoc says three times that the value "may vary from 0.0 to 1.0": in the class javadoc, the constructor @param and getValue. Nothing in Lucene keeps the value in that range, and nothing should: the grammar and the field-boost processor store the user's number unchanged, BoostQuery sets no upper bound, and the parser's own documentation uses jones^2 and (a OR b NOT c)^2.5 as examples. StandardSyntaxParser.parse("jones^2", "f") returns a node whose getValue() is 2.0.
Parsing, query building and scoring are correct. The only people misled are developers who extend the parser with their own processors, builders or validators and take the stated range at face value. The text has been there since the flexible parser was added in 2009 (LUCENE-1567) and is in every release since 2.9.0. The fix is three javadoc lines.
- The text.
BoostQueryNode.java:29"The boost value may vary from 0.0 to 1.0.";:39"@param value the boost value, it may vary from 0.0 to 1.0";:67"Returns the boost value. It may vary from 0.0 to 1.0." - The node checks nothing. The constructor (
:41-51) checks only that the child is not null and storesthis.value = value(:47);getValuereturns it (:71-73). - Producer 1, the grammar.
Boost ::= '^' <NUMBER>(StandardSyntaxParser.jj:692) returnsnew BoostQueryNode(node, parseFloat(boost))(:701), andparseFloatisFloat.parseFloat(token.image)(:115-116): no clamp, no range check. - Producer 2, field boosts. For a field node whose parent is not itself a field node,
BoostQueryNodeProcessorreads that field's configured boost and, if there is one, returnsnew BoostQueryNode(node, boost)with the value unchanged (BoostQueryNodeProcessor.java:44-57); the map comes fromStandardQueryParser.setFieldsBoost(StandardQueryParser.java:526). - Consumer.
BoostQueryNodeBuilderreturnsnew BoostQuery(query, boostNode.getValue())(BoostQueryNodeBuilder.java:48).BoostQueryrejects only non-finite or negative boosts (BoostQuery.java:40-41, "boost must be a positive float"; it compares withFloat.compare, so-0.0fcounts as negative and+0.0fpasses) and documents that values greater than one give more importance (:23-25). - So every boost the standard syntax accepts (an unsigned decimal number,
StandardSyntaxParser.jj:160) reaches the node unchanged:jones^2gives 2.0,smith^0.5gives 0.5,(a OR b)^2.5gives 2.5,jakarta^4gives 4.0, andsetFieldsBoost({b=5, t=10})builds(b:one)^5.0 (t:one)^10.0. Negative and non-finite values also reach the node and are rejected only whenBoostQueryis built (a field boost of -1; a literal beyondFloat.MAX_VALUE, which parses to Infinity). That matters for the fix wording.
Exposure and likelihood119 words
- Who. Code that reads
BoostQueryNode.getValue(): customQueryNodeProcessors,QueryBuilders or validators plugged intoStandardQueryParser. If it trusted the javadoc it could clamp boosts to 1.0 (silently turningjones^2intojones^1) or reject the parser's own documented example. - Who not. Applications that only call
StandardQueryParser.parse: the boost reachesBoostQueryunchanged and scoring is correct. - Default vs opt-in. The javadoc is what everyone sees, and the documented example hits the contradiction; acting on it needs custom code, which is opt-in and rare.
- Scale. None: it is static text.
- Why fix it anyway. A public class states a range that its only producers in Lucene violate, in every release since 2.9.0; the fix is three javadoc lines.
One JUnit test, one method, public API, deterministic: parse jones^2 with StandardSyntaxParser and check the node's value against the documented range 0.0 to 1.0.
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.lucene.queryparser.flexible.core.nodes;
import org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser;
import org.apache.lucene.tests.util.LuceneTestCase;
public class TestBoostQueryNodeDocumentedRange extends LuceneTestCase {
public void testStandardQueryParserExampleBoostIsInDocumentedRange() throws Exception {
BoostQueryNode node = (BoostQueryNode) new StandardSyntaxParser().parse("jones^2", "f");
// javadoc: "The boost value may vary from 0.0 to 1.0"; StandardQueryParser example jones^2: 2.0
assertTrue(
"expected a boost value in [0.0, 1.0], but getValue() = " + node.getValue(),
node.getValue() >= 0f && node.getValue() <= 1f);
}
}
cp $REPRO/TestBoostQueryNodeDocumentedRange.java $LUCENE/lucene/queryparser/src/test/org/apache/lucene/queryparser/flexible/core/nodes/
cd $LUCENE
./gradlew :lucene:queryparser:test --tests org.apache.lucene.queryparser.flexible.core.nodes.TestBoostQueryNodeDocumentedRange -Ptests.seed=DEADBEEF -Ptests.asserts=true
# assertions off: -Ptests.asserts=false (same failure)
- Expected
- Per the javadoc: a boost value in [0.0, 1.0].
- Observed
AssertionError: expected a boost value in [0.0, 1.0], but getValue() = 2.0, with assertions on and off.
More files, environment, full output218 words
- Deterministic. No randomness is involved; the seed is given only because the test framework wants one. Assertions on and off fail the same way.
- It transcribes the current javadoc. It cannot pass after the javadoc fix and is evidence for it, not a regression test.
- Regression form: the same test with the expectation the fixed javadoc gives,
assertEquals(2f, node.getValue(), 0f);in place of theassertTrue(...), passes on the fix with assertions on and off. - Releases. Equivalent tests (a JUnit, or a
main()for old releases) give 2.0 on every release run from 2.9.0 to 10.6.0; 2.4.1 has no flexible parser.
| lucene | main 645d4fe14a20 (11.0.0-SNAPSHOT); equivalent tests fail on every release run from 2.9.0 to 10.6.0 |
|---|---|
| jdk | OpenJDK 25.0.3 (build 25.0.3+9-LTS) |
| os | Linux, aarch64 |
cp $REPRO/TestBoostQueryNodeDocumentedRange.java $LUCENE/lucene/queryparser/src/test/org/apache/lucene/queryparser/flexible/core/nodes/
cd $LUCENE
./gradlew :lucene:queryparser:test --tests org.apache.lucene.queryparser.flexible.core.nodes.TestBoostQueryNodeDocumentedRange -Ptests.seed=DEADBEEF -Ptests.asserts=true
# assertions off: -Ptests.asserts=false (same failure)
# gradle, main 645d4fe14a20, seed DEADBEEF, -Ptests.asserts=true
TestBoostQueryNodeDocumentedRange > testStandardQueryParserExampleBoostIsInDocumentedRange FAILED
java.lang.AssertionError: expected a boost value in [0.0, 1.0], but getValue() = 2.0
at __randomizedtesting.SeedInfo.seed([DEADBEEF:D82E118751663037]:0)
at junit@4.13.2/org.junit.Assert.fail(Assert.java:89)
at junit@4.13.2/org.junit.Assert.assertTrue(Assert.java:42)
at org.apache.lucene.queryparser.flexible.core.nodes.TestBoostQueryNodeDocumentedRange.testStandardQueryParserExampleBoostIsInDocumentedRange(TestBoostQueryNodeDocumentedRange.java:27)
:lucene:queryparser:test (FAILURE): 1 test, 1 failure
# -Ptests.asserts=false: the same failure
# javadoc fix (option A): checkGoogleJavaFormat, ecjLintMain, renderJavadoc pass; the flexible-parser tests pass (162 tests, 1 skipped);
# the same test expecting 2.0 passes with assertions on and off- The same module's syntax docs use
jones^2and(a OR b NOT c)^2.5; the classic syntax page usesjakarta^4. BoostQuery, which the node becomes, says values above one give more importance and sets no upper bound.- Upstream tests assert boosts of 2.0, 5.0 and 10.0 on queries built through this node; the 2009 commit that wrote the sentence tested
term^2.
The full argument244 words
- The same module documents boosts above 1.
StandardQueryParser.java:161-173("Term boosting"):jones^2 OR smith^0.5andfield:(a OR b NOT c)^2.5 OR field:d. The classic syntax page: "The higher the boost factor, the more relevant the term will be" (classic/package-info.java:250-251),jakarta^4 apache(:261), and "By default, the boost factor is 1. Although the boost factor must be positive, it can be less than 1 (e.g. 0.2)" (:268-269). - The query it builds says so too.
BoostQuery.java:23-25: "values that are greater than one will give more importance to the scores returned by this query"; the constructor sets no upper bound and rejects only negative values (-0.0ftoo), NaN and infinities (:40-41). - Upstream tests.
TestQPHelper.java:974-976("hello"^2.0buildsBoostQuerywith boost 2.0) andTestMultiFieldQPHelper.java:154-166(testBoostsSimple: field boosts 5 and 10, asserted as(b:one)^5.0 (t:one)^10.0); both check the built query, not the node itself. No test asserts a value in [0, 1] forBoostQueryNode; no queryparser test names the class. - History. The commit that wrote the three sentences,
343992fcbb4(LUCENE-1567, 2009), also added tests assertingterm^2givesterm^2.0(with^3and^4cases and field boosts 5 and 10), and the query syntax page of that time already saidjakarta^4. The saved history and prior-art searches found no commit, issue or list thread that discusses or defends a [0, 1] range; a 2025 clean-up of this file (#14737) left the line untouched without comment.
The devil's-advocate check134 words
The strongest case for "not a bug": "may vary" is permissive, a loose hint rather than a contract; or the range is a caller precondition of a generic core node that some other syntax could honour. The intended-behaviour checker also tried "normalized-boost design", "accepted / won't fix" and "unreachable".
Why it fails: the text names a closed range, and the only producers in Lucene (the only new BoostQueryNode sites in the source tree), the standard grammar and BoostQueryNodeProcessor, put values above 1 into the node as designed, starting with the module's own documented example. A hint that is wrong for every value above 1 the documented syntax produces is still wrong, and no saved search found a discussion defending it. Verdict: a documentation defect; the code is right and should not change.
Recommended A. State what the value means once; drop the range from @param and getValue (recommended)
Javadoc only. Say that boost values above 1.0 raise the score and values below 1.0 lower it (jones^2 gives 2.0), and that the node does not check the value while BoostQuery rejects negative and non-finite boosts; drop the range from @param and getValue. Validated: format, lint and javadoc checks and the flexible-parser tests pass.
All options and trade-offs3 options · 725 words
Three options, all javadoc-only; option A was applied in a scratch worktree of main 645d4fe14a20 and checked with gradle. Same file as LFV-975 (toQueryString prints small boosts in E-notation).
A. State what the value means once; drop the range from @param and getValue (recommended) (validated by the checks)
Say that values above 1.0 raise the score and values below 1.0 lower it, with the documented example, and that the node does not check the value while BoostQuery rejects negative and non-finite boosts.
--- a/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/nodes/BoostQueryNode.java
+++ b/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/nodes/BoostQueryNode.java
@@ -26,7 +26,9 @@ import org.apache.lucene.queryparser.flexible.messages.MessageImpl;
* A {@link BoostQueryNode} boosts the QueryNode tree which is under this node. So, it must only and
* always have one child.
*
- * <p>The boost value may vary from 0.0 to 1.0.
+ * <p>Boost values above 1.0 raise the score of the boosted query and values below 1.0 lower it; for
+ * example {@code jones^2} gives a value of 2.0. This node does not check the value: {@link
+ * org.apache.lucene.search.BoostQuery} rejects negative and non-finite boosts.
*/
public class BoostQueryNode extends QueryNodeImpl {
@@ -36,7 +38,7 @@ public class BoostQueryNode extends QueryNodeImpl {
* Constructs a boost node
*
* @param query the query to be boosted
- * @param value the boost value, it may vary from 0.0 to 1.0
+ * @param value the boost value
*/
public BoostQueryNode(QueryNode query, float value) {
if (query == null) {
@@ -64,7 +66,7 @@ public class BoostQueryNode extends QueryNodeImpl {
}
/**
- * Returns the boost value. It may vary from 0.0 to 1.0.
+ * Returns the boost value.
*
* @return the boost value
*/Trade-offs
- It does not imply that the node validates anything: the node stores whatever it gets, and
BoostQueryis where negative and non-finite boosts are rejected. - It adds a cross-module
{@link}from a core node toorg.apache.lucene.search.BoostQuery(the standard package's javadoc already linksorg.apache.lucene.search.Query). A custom builder that builds something else is not covered by that sentence; "This node does not check the value." alone also works. - Option C (delete the range, add nothing) makes no new claim and is the most conservative alternative. "Negative" in the last sentence means what
BoostQuerychecks (Float.compare, so-0.0fis rejected too); "This node does not check the value." alone avoids the point. - Backport: not tested on a maintenance branch.
BoostQueryNode.javais byte-identical (the same git blob) on main, at the 10.6.0 tag and onbranch_10_6, so the diff applies as is; the checks were not run there.
What the checks validated
- Gradle, scratch worktree of
645d4fe14a20, JDK 25.0.3, seedDEADBEEF::lucene:queryparser:checkGoogleJavaFormat,:lucene:queryparser:ecjLintMain(its settings make invalid javadoc an error:EcjLintPlugin.java:52loadsecj.javadocs.prefs, which setsinvalidJavadoc=error) and:lucene:queryparser:renderJavadocpass, and the renderedBoostQueryNode.htmlshows the new class text,value - the boost valueand "Returns the boost value." - The flexible-parser tests (
org.apache.lucene.queryparser.flexible.*: the 14 upstream classes incl.TestQPHelperandTestMultiFieldQPHelper, plus the minimal test in its regression form expecting 2.0, 15 classes in all) pass: 162 tests, 1 skipped (the@IgnoreatTestQPHelper.java:928), 0 failures, assertions on; the regression form passes with assertions off too. - The first run did not record the scratch worktree's HEAD. An independent fact check applied option A to a fresh worktree of
645d4fe14a20and re-rancheckGoogleJavaFormat,ecjLintMainand the regression form: all pass. - The minimal test as written (the old range) was not re-run with the fix: the fix changes only comments, so it keeps failing as on stock main, by design.
The hunt's first sketch: the class sentence becomes "The boost value is a non-negative float: values above 1.0 raise the score of the boosted query, values below 1.0 lower it", and @param and getValue say "a non-negative float".
Trade-offs
- Reads as if the node guaranteed non-negativity, which it does not: a field boost of -1 reaches the node and fails only when
BoostQueryis built.
What the checks validated
Not run (only patch --dry-run, which applies to main).
Drop "The boost value may vary from 0.0 to 1.0.", and leave @param value the boost value and "Returns the boost value."
Trade-offs
- Smallest change and correct, but loses the hint about what values mean.
What the checks validated
Not run (option A makes the same @param and getValue change).
Code outside Lucene that extends the flexible parser and trusts this range could clamp or reject valid user boosts; the Lucene.NET port copies the same sentence.
Non-formal: code reading. While a hunt unit checked how the flexible parser prints numbers, the finder read BoostQueryNode's range, compared it with the parser's documented examples and confirmed it with a public-API JUnit on the stock main jars.
The method in full100 words
- Lead. The unit swept the flexible parser's number printing (
toQueryStringsiblings of LFV-892). While modelling the boost printer, the finder saw the javadoc range; the cross-model review of the sibling printing bug had also leaned on it. - Test. A public-API JUnit parsed the documented example
jones^2and checked the node value against the stated range: 1 of 1 fails on main with assertions on and off, on branch_10_64ae946e5a9b, 10.5.2 and 9.12.3. - Verification. An independent reproducer, a cross-model refute review (OpenAI GPT-5.6), an intended-behaviour devil's advocate, an adjudication and a credibility audit followed; all agree on the facts.
No prior report found in the recorded searches (2026-10-28, 115 query records; rechecked 2026-11-01): GitHub issues and PRs incl. the JIRA archive, commits, mailing lists, downstream trackers, the web and every open PR. The nearest issues (#6938, #7660, #6554) are other BoostQueryNode bugs.
Searches4 searches
- The exact phrase "may vary from 0.0 to 1.0" has 0 issue, PR, JIRA, dev@, java-user@ or issues@ hits; only 3 commits@ diff notifications from 2012 show it as context.
- Nearest items, other bugs: #6938 (LUCENE-5876, NPE in
BoostQueryNodeBuilder), #7660 (LUCENE-6602, null child intoString), #6554 (LUCENE-5491, exception inBoostQueryNodeProcessor), #2641 (LUCENE-1567, the origin; none of its 119 comments discusses a boost range), #14737 (merged 2025-06-04, edited this file and kept the text, approved without comments). - Recheck 2026-11-01: the 63 issues and PRs updated since 2026-10-28 were listed; none mentions
BoostQueryNodeor the phrase, and of the 52 PRs only #16853 (EscapeQuerySyntaxImpl) touches the flexible parser. Upstream mainf815e679bf6cstill has the text. - Open one-line javadoc PRs, for example #16949 (the surround parser), do not touch this file.
| Where | When | How | Found |
|---|---|---|---|
| GitHub issues, PRs and commits (apache/lucene, incl. migrated JIRA) | 2026-10-28 | a scripted prior-report search (53 queries over all the sources in this table) plus 35 manual issue/PR searches, 4 code searches and 6 other GitHub API reads: the exact phrase, "vary from 0.0", boost "0.0 to 1.0", "between 0 and 1" boost, boost "greater than 1", BoostQueryNode, BoostQueryNodeProcessor, BoostQueryNodeBuilder; the 119 comments of #2641 | no report |
| Open PRs | 2026-10-28 and 2026-11-01 | changed files of all 414 open PRs (2026-10-28); changed files of the 52 PRs updated since 2026-10-28 (2026-11-01) | none touches flexible/core/nodes |
| Mailing lists, JIRA archive, downstream trackers, web | 2026-10-28 | dev@, java-user@, issues@, commits@ (plus 12 manual list-statistics queries); the LUCENE and SOLR JIRA archive (plus 4 JIRA REST queries); Elasticsearch, OpenSearch, Solr and Lucene.NET trackers; 8 web queries | no report (Lucene.NET copies the text; no issue there) |
| Recent issues and PRs | 2026-11-01 | the 63 issues and PRs updated since 2026-10-28, titles and bodies | nothing about it |
Written in the flexible parser's first commit, 343992fcbb4 (LUCENE-1567, 2009-08-03), and first released in 2.9.0. Every version run, 2.9.0 to 10.6.0 and main, gives 2.0 for jones^2. Not a regression, and not fixed upstream.
Versions tested6 rows
- Introduced:
343992fcbb4b31249f07354014723f18d0508d8a, 2009-08-03, "LUCENE-1567: New flexible QueryParser framework." (JIRA LUCENE-1567, now #2641; a JIRA patch by Luis Alves and Adriano Campos, committed by Michael Busch). Found withgit blame -C -C -M(lines 29 and 39 pass through the 2021 reflow2cbf261032d; blame at its parent gives343992fcbb4for all three) andgit log -S 'vary from 0.0 to 1.0'. - By reading: all 149
lucene-queryparserversions on Maven Central (147 final releases 2.9.0 to 10.5.2 plus 4.0.0-ALPHA and BETA), the 10.6.0 tag and main carry the three sentences. - Not a regression: no version ever matched the text. The
jones^2and^2.5examples inStandardQueryParser's javadoc came later (#429, LUCENE-10223, 9.1.0).
| Version or commit | Verdict | Result |
|---|---|---|
main 645d4fe14a20 | affected | minimal test fails, gradle and standalone, assertions on and off |
10.6.0 (4ae946e5a9b), 10.5.2, 10.0.0 | affected | equivalent JUnit fails (10.6.0 with assertions off, also on the released jars from Maven Central, 10.5.2 on and off, 10.0.0 on) |
| 9.12.3, 9.1.0, 9.0.0 | affected | JUnit fails on 9.12.3 and 9.1.0; main() fails on 9.0.0 and 9.12.3 |
| 8.11.4, 4.0.0, 3.6.2 | affected | main() fails: getValue() = 2.0 |
| 2.9.0 | first release | main() fails: getValue() = 2.0; control smith^0.5 = 0.5 |
| 2.4.1 | no API | compile error: no flexible query parser |
Every check agrees on the facts: the finder, an independent reproducer, an intended-behaviour devil's advocate, the adjudication (confirmed, high, documentation defect) and a credibility audit. A cross-model reviewer (OpenAI GPT-5.6) rated it real; its refute-lens run said not real only because it could execute nothing.
Checks and verdicts316 words
Each check is listed with its verdict.
| Check | Verdict | What it found |
|---|---|---|
| finder | reproduced | Public-API JUnit fails 1 of 1 on main (assertions on and off), branch_10_6, 10.5.2 and 9.12.3. |
| cross-model review, intended lens | real, high | An independent reviewer from another model family (OpenAI GPT-5.6), read-only: intended argument weak, root cause agrees, the fix fixes it. |
| independent reproducer | real | Own test: jones^2 OR smith^0.5 gives [2.0, 0.5], setFieldsBoost({a=5}) gives 5.0; fails 2 of 3 (the sanity check passes) on main, 10.5.2 and 9.12.3. |
| cross-model review, refute lens | not real, low, procedural only | Could run nothing; 9 of its 10 evidence items support real, and it wrote that the static evidence "strongly favors a genuine, low-impact documentation defect". Its proposed check, run separately, fails 2 of 3 (3.25, 7.0; the 0.75 control passes). |
| intended-behaviour devil's advocate | real | Five intended readings fail; its fact probe passes on stock main (2.0, 0.5, 2.5, 4.0; field boosts 5 and 10; -1 and Infinity rejected only by BoostQuery). |
| adjudication | confirmed, high, doc, minor | Re-ran the independent reproduction's test (3 run, 2 failures) and the finder's (1 run, 1 failure) with new seeds. |
| credibility audit | confirmed, minor | Re-ran the finder's test; a committer would merge the three-line javadoc fix without discussion and see it as a doc clean-up. |
| this brief (2026-11-01) | validated | Gradle in a scratch worktree: the minimal test fails on stock main; with option A the checks, the flexible-parser tests and the regression form pass. |
| fact checks of the brief (2026-11-01) | facts hold; wording fixed | An independent fact check (it re-ran the gradle command, the standalone run on main and on the released 10.6.0 jars, and option A's checks) and a cross-model fact check (OpenAI GPT-5.6, read-only) confirmed the bug and the main facts and found wording errors and overclaims, which are corrected here. |
The diagram as a still image, generated with AI and checked against the code. It is under 10 MB, the limit for a GitHub attachment.
- PNG 427 KB
Finding record, Lucene and upstream sources, repro files and diagram notes1,083 words
- Registry title
- Flexible query parser: the BoostQueryNode javadoc says three times (class, constructor @param, getValue) that the boost value 'may vary from 0.0 to 1.0', but boosts above 1.0 are standard syntax and are stored unchanged: StandardQueryParser's own documented example jones^2 gives a node value of 2.0 (documentation defect; every release since 2.9.0)
- Area
- queryparser / flexible core nodes (BoostQueryNode javadoc)
- Priority
- Low, rank 1276: The wrong [0, 1] boost range in the BoostQueryNode javadoc only misleads developers who extend the niche flexible query parser while parsing and scoring stay correct.
- Impact class
- minor
- Impact note
- Documentation only: parsing, query building and scoring are correct in every configuration; the people misled are developers who extend the flexible query parser (custom processors, builders or validators that read BoostQueryNode) and trust the stated [0, 1] range, who could then clamp or reject the valid boosts the parser itself produces (jones^2 gives 2.0, a setFieldsBoost value of 5 gives 5.0); the text has been wrong in every release since 2.9.0 (2009), and no scale or special setting is needed to see it.
- Summary
- You read the javadoc of the flexible query parser's BoostQueryNode, which says in the class doc, the constructor @param and getValue that the boost value 'may vary from 0.0 to 1.0', and expect the parser to put only such values into the node. Instead boosts above 1.0 are ordinary syntax and pass through unchanged: StandardQueryParser's own javadoc examples jones^2 and (..)^2.5 give nodes with values 2.0 and 2.5, field boosts set with setFieldsBoost (5, 10) do the same, and BoostQuery accepts any finite boost >= 0. The code is right and the javadoc is wrong; nothing clamps or checks the range, and nothing should.
- Introduced
- 343992fcbb4b31249f07354014723f18d0508d8a (Subversion lucene/java/trunk r800191, 'LUCENE-1567: New flexible QueryParser framework.', JIRA LUCENE-1567, migrated to GitHub as #2641; a JIRA patch by Luis Alves and Adriano Campos, committed by Michael Busch; there is no pull request). It created BoostQueryNode with all three sentences, while its own TestQPHelper already asserted term^2 -> term^2.0 and the query syntax page documented 'jakarta^4 apache', so the text never matched the code and this is not a regression [read: git blame -C -C -M (lines 29 and 39 go through the 2021 reflow 2cbf261032d, LUCENE-9570; blame at 2cbf261032d^ gives 343992fcbb4 for all three) and git log -S 'vary from 0.0 to 1.0' in the full-history clone; exec: 2.9.0 already fails] (2009-08-03 (author and commit date of 343992fcbb4))
- First affected version
- 2.9.0 [exec] (the first release with the flexible query parser; 2.4.1 has no such API: compile_error [exec], and no lucene-queryparser artifact exists for it)
- Affected versions
- every lucene-queryparser version on Maven Central, 147 final releases 2.9.0 to 10.5.2 plus 4.0.0-ALPHA/BETA [read: the three sentences in each release tag or SHA-1-checked sources jar], plus 10.6.0 (branch_10_6 4ae946e5a9b), branch_10x and main; not fixed upstream (upstream main 645d4fe14a20 still has the text). Executed [exec]: 2.9.0, 3.6.2, 4.0.0, 8.11.4, 9.0.0, 9.12.3, 10.5.2 and main with a main() variant; 9.1.0, 9.12.3, 10.0.0, 10.5.2 and main with the JUnit (-ea; the JUnit does not compile on 9.0.0); 10.5.2, 9.12.3, branch_10_6 4ae946e5a9b and main with the JUnit (-da). Every run gives getValue() = 2.0 for jones^2, and the in-range control smith^0.5 = 0.5 passes everywhere.
- Status
- documented (brief, checks and a per-finding doc exist)
- Verification
- Confirmed by AI-run checks: an independent repro, a cross-model review that could only read the code (its low-confidence “not real” is procedural), an intended-behaviour check, a prior-report search and a skeptical audit; not yet triaged by a person.
- Found by
- non-formal: code reading
- Upstream and prior reports
- none found (searched 2026-10-28): a scripted prior-report search with 53 queries (16 GitHub issue/PR, 3 commit, 2 downstream, 20 mailing-list over dev@, java-user@, issues@ and commits@, 4 JIRA-archive, 8 web; no close match) plus 35 manual GitHub searches and 4 code searches (115 query records in all); the exact phrase 'may vary from 0.0 to 1.0' has 0 issue, PR, JIRA or dev@/java-user@/issues@ hits (only 3 commits@ diff notifications from 2012 show it as context); all 414 open apache/lucene PRs checked by changed files, none touches lucene/queryparser/.../flexible/core/nodes. Nearest items, not the same bug: #6938 (LUCENE-5876, NPE in BoostQueryNodeBuilder), #7660 (LUCENE-6602, null child in BoostQueryNode.toString), #6554 (LUCENE-5491, exception in BoostQueryNodeProcessor), #2641 (LUCENE-1567, the origin; none of its 119 comments discusses a boost range) and #14737 (merged 2025-06-04, a node-class clean-up that edited this file and kept the text, approved without comments). Re-checked (2026-10-28): 'BoostQueryNode' still finds only #6938 and #7660; '"vary from 0.0 to 1.0"' finds nothing; none of the 22 issues and PRs created since 2026-10-27 is about it; none of the 45 PRs updated since 2026-10-27 touches flexible/core/nodes; the file's newest upstream commit is still f2493e586b1 (#14737, author date 2025-06-04), and upstream main still has the three sentences at lines 29, 39 and 67.
- Lucene commit tested
- main
645d4fe14a20(2026-10-28): the minimal test through gradle in a scratch worktree with assertions on and off, then the javadoc fix with the format, ecj lint and javadoc checks, the flexible-parser tests and the regression form of the test (2026-11-01); standalone, the minimal test on main and equivalent tests on releases 2.9.0 to 10.6.0 (2.4.1 has no such API). Upstream mainf815e679bf6c(2026-11-01) has the same text.
- Finding record
lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/nodes/BoostQueryNode.java:29 (at a9779ef3c193) ('The boost value may vary from 0.0 to 1.0.'), :39 ('@param value the boost value, it may vary from 0.0 to 1.0'), :67 ('Returns the boost value. It may vary from 0.0 to 1.0.')lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/StandardQueryParser.java:161-172 (at a9779ef3c193) ('Term boosting': jones^2 OR smith^0.5; field:(a OR b NOT c)^2.5 OR field:d)lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/parser/StandardSyntaxParser.jj:696-702 (at a9779ef3c193) (Boost: '^' <NUMBER> -> new BoostQueryNode(node, parseFloat(boost)), no range check) and standard/processors/BoostQueryNodeProcessor.java:57 (field boost wrapped unchanged)lucene/core/src/java/org/apache/lucene/search/BoostQuery.java:38-44 (at a9779ef3c193) (rejects only non-finite or negative boosts)lucene/queryparser/src/test/org/apache/lucene/queryparser/flexible/standard/TestQPHelper.java:974-976 (at a9779ef3c193) ('"hello"^2.0' -> BoostQuery 2.0) and TestMultiFieldQPHelper.java:154-166 (testBoostsSimple: field boosts 5 and 10)SECURITY.md 'In-scope reports' and 'Reporting out-of-scope issues publicly' (cited by the security record)
- The finding in 30 seconds
lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/nodes/BoostQueryNode.java:29, :39, :41-51, :47, :67, :71-73lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/parser/StandardSyntaxParser.jj:115-116, :160, :692, :701lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/processors/BoostQueryNodeProcessor.java:44-57lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/builders/BoostQueryNodeBuilder.java:48lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/StandardQueryParser.java:161-173, :526lucene/core/src/java/org/apache/lucene/search/BoostQuery.java:23-25, :38-44
- Why it looks unintended
lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/StandardQueryParser.java:161-173lucene/queryparser/src/java/org/apache/lucene/queryparser/classic/package-info.java:250-251, :261, :268-269lucene/core/src/java/org/apache/lucene/search/BoostQuery.java:23-25, :40-41lucene/queryparser/src/test/org/apache/lucene/queryparser/flexible/standard/TestQPHelper.java:974-976lucene/queryparser/src/test/org/apache/lucene/queryparser/flexible/standard/TestMultiFieldQPHelper.java:154-166
- TestBoostQueryNodeDocumentedRange.java:
repro/LFV-973/src/org/apache/lucene/queryparser/flexible/core/nodes/TestBoostQueryNodeDocumentedRange.java(31 lines)
assets/LFV-973/bug.png(427 KB)
