MathUtil.asinh, acosh and atanh return Infinity, 0 or the wrong infinity for very large, tiny or large negative arguments

MathUtil.asinh, acosh and atanh evaluate the textbook formulas log(sqrt(a*a + 1) + a), log(sqrt(a*a - 1) + a) and 0.5 * log((1 + a) / (1 - a)) with no range reduction and no domain check. For every finite |a| of 2^512 (about 1.34e154) or more, a*a overflows, so asinh(1e200) and acosh(1e200) return Infinity instead of 461.2101657793691. For tiny arguments the log's argument (1 + a for asinh, (1 + a) / (1 - a) for atanh) rounds to exactly 1, so asinh(1e-17) and atanh(1e-17) return 0.0 instead of 1e-17, and just above that range asinh keeps only about 8 correct digits at 1e-8. acosh never checks its documented rule "If the argument is less than 1, then the result is NaN": acosh(-1e8) returns -Infinity and acosh(-1e200) Infinity. The default expression functions asinh, acosh and atanh are bound straight to these methods, so expression scores, sort keys and filters come out wrong with no error; outside tests, no other Lucene code calls them. Every release since 4.6.0 (2013) is affected; the inputs are extreme (plausible paths are sentinels and malformed or extreme data), so the realistic exposure looks low. A range-reduced implementation with an explicit NaN for acosh below 1 fixes all of it.

1. asinh (MathUtil.java:102-113) strips the sign and returns sign * Math.log(Math.sqrt(a * a + 1.0d) + a) (:112). For every finite a >= 2^512, a * a is +Infinity, and sqrt and log keep it: the result is Infinity (-Infinity for a negative argument). For 0 < a <= 2^-53, the log's argument Math.sqrt(a * a + 1.0d) + a is 1.0 + a, which rounds to 1, and log(1) is 0. (1 + a * a is already exactly 1 for every a up to about 1.05e-8, where asinh is still nonzero, so the value is lost at 1 + a.) Both thresholds are exact: SMT over IEEE binary64 (cvc5 and z3, QF_FP) proves that the log argument is Infinity for every finite a >= 2^512 and not infinite on [1, 2^512), and that it is exactly 1 on (0, 2^-53] and not 1 above. Just above the small threshold the relative error is about 1.1e-16 / a: 6.7e7 ulp at 1e-8, 237 ulp at 1e-3.
2. acosh (:127-129) is one line, return Math.log(Math.sqrt(a * a - 1.0d) + a); (:128), with no check of its documented domain (:124). Large positive arguments overflow like asinh. Negative arguments reach NaN only by accident (for -10, sqrt(99) - 10 is negative, and log of a negative number is NaN). From about -6.7e7 (-2^26) down, rounding makes sqrt(a * a - 1) + a come out 0 or negative (from about -1.34e8 on, a * a - 1 rounds to a * a): between about -1.34e8 and -6.7e7 the result is -Infinity for some arguments and NaN for others, from about -1.34e8 (-2^27) to -2^512 it is -Infinity, and from -2^512 down a * a overflows and the result is Infinity. No negative argument gives a finite result.
3. atanh (:144-154) returns mult * Math.log((1.0d + a) / (1.0d - a)) (:153). For 0 < |a| <= 2^-54 (about 5.6e-17) both 1 + a and 1 - a round to 1, so the result is 0. There is no overflow case (the domain is |a| <= 1).
4. Expressions. JavascriptCompiler.properties:23, :25, :28 bind acosh, asinh and atanh to these methods (while cosh, sinh and tanh go to java.lang.Math, :31, :43, :46); compile(String) uses this default function set (JavascriptCompiler.java:154-155), and the package javadoc lists them (js/package-info.java:31-32). Outside tests, nothing else in Lucene calls them.

• Who. Direct callers of the public static MathUtil.asinh/acosh/atanh (public final class MathUtil, no @lucene.internal), and users of the optional expressions module whose expressions call asinh, acosh or atanh with the default function set over literals or bound values such as doc values or _score. Outside tests, no Lucene code calls them.
• Sort ties. Sorting by the expression asinh(x) over a DoubleDocValuesField gives the keys 346.0809111296668 Infinity Infinity for x = 1e150, 1e160 and 1e200, so two distinct documents tie although asinh is strictly increasing (correct: 346.08, 369.11, 461.21). Executed on main with assertions on and off, and on 10.5.2.
• Scores. With FunctionScoreQuery and _score + 0.000001 * asinh(x) over a term query, the documents with x = 1e200 and 1e160 score Infinity and rank first and second, ahead of better text matches scoring 0.0498 and 0.0486; with correct math they score 0.0310 and 0.0309. Correct math caps the term at 710.48 times its weight.
• Filters. FunctionMatchQuery with asinh(x) > 400 also matches x = 1e160 (true asinh 369.1), acosh(x) > 1 matches x = -1e200 (true NaN), and asinh(x) == 0 matches x = 1e-17.
• Scope of these examples. The sort, score and filter results come from constructed public-API tests that show what can happen; no production impact was observed.
• What it takes. Magnitudes of at least 1.34e154 (asinh, acosh), at most 1.1e-16 (asinh) or 5.6e-17 (atanh), or acosh of about -6.7e7 or less. These ranges look contrived for typical ranking inputs; sentinels and malformed or extreme data are plausible paths. The tiny-argument zero changes a result by at most about 1.11e-16 (asinh) or 5.6e-17 (atanh). No public report was found in the recorded searches. Registry likelihood contrived, priority low.
• Not affected. Nothing is written to the index; there is no exception or hang, and no sign of resource exhaustion (30 million evaluations on collapse inputs took 111 ms in one local run; not a performance claim).

The class sits in org.apache.lucene.util only to be next to TestMathUtil. It compiles on 9.1 and later (org.apache.lucene.tests.util.LuceneTestCase); on 9.0 and older the base class is org.apache.lucene.util.LuceneTestCase.

It shows the overflow only. The other symptoms (acosh overflow, the tiny-argument zeros, acosh of large negative arguments) are the executed values in the table above; a NaN assertion for acosh (for example assertTrue(Double.isNaN(MathUtil.acosh(-1e8))), not run as a test) would be a second test method with its own fix hunk.

Regression test: this same test, as written.

cp $REPRO/TestMathUtilAsinhLargeArgument.java $LUCENE/lucene/core/src/test/org/apache/lucene/util/
cd $LUCENE && ./gradlew :lucene:core:test --tests org.apache.lucene.util.TestMathUtilAsinhLargeArgument -Ptests.seed=DEADBEEF
# assertions off fails the same way: add -Ptests.asserts=false (the recorded runs added --offline --no-daemon)

# gradle, scratch worktree of 645d4fe14a20, seed DEADBEEF, assertions on
TestMathUtilAsinhLargeArgument > testAsinhOfLargeFiniteArgument FAILED
    java.lang.AssertionError: expected:<461.2101657793691> but was:<Infinity>
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:667FD9FA2387ECB5]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.failNotEquals(Assert.java:835)
        at org.junit.Assert.assertEquals(Assert.java:555)
        at org.junit.Assert.assertEquals(Assert.java:685)
        at org.apache.lucene.util.TestMathUtilAsinhLargeArgument.testAsinhOfLargeFiniteArgument(TestMathUtilAsinhLargeArgument.java:25)

# assertions off (-Ptests.asserts=false): the same message
# standalone runner, seed DEADBEEF: 1 of 1 failed, same message, on main 645d4fe14a20 (-ea and -da), 10.6.0, 9.12.3
# with the recommended fix: the test passes (assertions on and off)

• The contract. "Calculates inverse hyperbolic sine / cosine / tangent of a double value" (MathUtil.java:91, :116, :132), followed by special cases: an infinite result for an infinite argument (asinh), for +Infinity (acosh) or for +-1 (atanh), and a zero for a zero argument (asinh, atanh) or for +1 (acosh) (:93-100, :118-125, :134-142). The true values here are finite and nonzero, so Infinity at 1e200 or 0 at 1e-17 is the loss of the whole value, not rounding. (The list has no "only"; the argument rests on what the functions are.)
• The acosh rule. "If the argument is less than 1, then the result is NaN" (:124). TestMathUtil.testAcoshMethod asserts NaN for 0.9, 0, -0, -0.9, -1, -10 and -Infinity (TestMathUtil.java:105-121).
• Monotonicity. asinh(nextDown(2^512)) is 355.58450362725193; the next double up gives Infinity.
• History. The formula lines blame to c73f175c71ee (LUCENE-5207, 2013) at 645d4fe14a20; the migrated LUCENE-5207 discussion (#6271, 89 JIRA comments) never discusses the range or accuracy of these functions. The upstream tests stop at |x| = 1234567.89 (TestMathUtil.java:123-126, :146-153, :177-182; TestJavascriptFunction.java:54-57, :74-81, :108-113), so nothing pins the current behaviour.
• Precedent. In LUCENE-4162 (#5234, 2012), about another public MathUtil method, Uwe Schindler wrote: "Otherwise this class must be hidden, we should not offer any public method to the users that can loop infinite without parameter checking." It was fixed.

The strongest case for "intended" (a devil's-advocate verifier, the cross-model reviews and the adjudication): the util package javadoc says "APIs in this package are primarily internal and may change between releases" (util/package-info.java:35); JavascriptCompiler is @lucene.experimental (JavascriptCompiler.java:111); no ulp bound is promised, unlike java.lang.Math's functions; the upstream tests accept an absolute error of 1e-6 (1e-7 in TestJavascriptFunction) at moderate magnitudes; and the code does exactly what its author wrote.

Why it fails: the first two are about API stability, not permission to return wrong values. The package sentence was added on 2026-02-18 by a docs-only change (#15719), MathUtil itself has no @lucene.internal, and the expression functions are documented for users. No tolerance accepts Infinity for a finite argument, and the acosh NaN rule is explicit. Where the defence is moderate: the tiny-argument zero (an absolute error of at most about 1.11e-16, inside the tests' tolerance) and the graded precision loss above it; for those, a fix is an accuracy improvement rather than a bug fix. The independent reproducer, the devil's-advocate verifier and the intended-lens review said real; the refute-lens review's "not real" was procedural (it could not run code and agreed with the root cause); the adjudication confirmed with high confidence.

The recommended fix was validated through gradle in a scratch worktree of 645d4fe14a20; the smaller guard-only option and documenting the limits were not built.

For large arguments use log(a) + ln 2 (above 2^28, sqrt(a * a +- 1) equals a to double precision); between 2 and 2^28 an equivalent form without overflow; near 0 (asinh, atanh) and near 1 (acosh) Math.log1p forms that avoid the cancellation; acosh returns NaN for any argument below 1 or NaN. The same reductions as fdlibm.

// asinh, after the sign is stripped
final double r;
if (a > 0x1p28) {
  r = Math.log(a) + LN2; // a * a may overflow
} else if (a > 2) {
  r = Math.log(2 * a + 1 / (Math.sqrt(a * a + 1) + a));
} else {
  final double t = a * a; // log1p avoids log(1 + tiny)
  r = Math.log1p(a + t / (1 + Math.sqrt(1 + t)));
}
return sign * r;

// acosh: if (!(a >= 1)) return Double.NaN; then the same three ranges
// atanh: return mult * Math.log1p(2 * a / (1 - a));

Trade-offs

• Last-bit changes. The formulas are rewritten for ordinary arguments too, so results can move by an ulp and an expression score with them; the upstream tests (absolute tolerance) still pass.
• Cost. One or two extra comparisons, and Math.log1p instead of Math.log for |a| <= 2 (asinh, acosh) and in every atanh call. Not benchmarked.
• Accuracy. Maximum error 1.04 ulp (asinh), 0.98 ulp (acosh) and 1.19 ulp (atanh) over 6,898 points covering every binade, against a 200-digit reference (a sample, not a proof of a global error bound), with 0 collapses (stock: 5,028, 1,682 and 3,342 collapses on the same points). fdlibm's split for atanh below 0.5 (0.5 * log1p(2a + 2a*a / (1 - a))) could bring atanh under 1 ulp; not built.
• Design support. Elasticsearch's own 2026 implementations of these functions use the same reductions.

What the checks validated

• Gradle, scratch worktree of 645d4fe14a20, OpenJDK 25.0.3, seed DEADBEEF (2026-10-31). Stock: the repro fails with assertions on and off. With the fix: it passes with assertions on and off.
• The repro plus TestMathUtil (its acosh test asserts the NaN cases): 7 tests, 0 failures. :lucene:expressions:test, the whole module including TestJavascriptFunction: 13 suites, 170 tests, 0 failures.
• :lucene:core checkGoogleJavaFormat, ecjLintMain, ecjLintTest (with the repro test in place), forbiddenApisMain, forbiddenApisTest: pass. The finder's original comment on the atanh line was 101 columns and failed the format check, so the diff shortens it; the code is unchanged.
• Earlier: the finder's 7 tests, an independent reproducer's 12 tests (including the acosh NaN cases) and TestMathUtil plus TestJavascriptFunction (35 tests) pass with it; the value probe gives the correct value for every row of the table above.

if (a > 0x1p28) return sign * (Math.log(a) + LN2); in asinh (the same without the sign in acosh), and if (!(a >= 1)) return Double.NaN; first in acosh; the formulas stay as they are otherwise.

Trade-offs

Fixes the Infinity collapse and the documented NaN rule. It leaves every asinh result for |a| <= 2^28 and every acosh result for 1 <= a <= 2^28 bit-for-bit unchanged (below 1, only the wrong acosh results change, to NaN), but keeps the tiny-argument zeros and the graded precision loss.

State in the javadoc that the functions are accurate only for moderate arguments.

Trade-offs

Leaves Infinity for finite input, and contradicts the acosh special case unless that line changes too.

• Code reading first. The hunt unit for numeric, string and version utilities chose the MathUtil methods an earlier study had not covered and read the formulas.
• SMT. cvc5 1.4.2 and z3 4.12.2, QF_FP (exact IEEE binary64 semantics with round-to-nearest-even): 6 UNSAT queries prove the exact collapse regions over all doubles (the log argument is +Infinity for every finite a >= 2^512 and not infinite on [1, 2^512); it is exactly 1 for every a in (0, 2^-53] for asinh and (0, 2^-54] for atanh, and not 1 above), with 3 SAT vacuity probes; both solvers agree.
• Measurement. An error table of the stock jar over 6,898 points covering every binade, against a 200-digit decimal reference.
• JUnit. Public-API tests on the stock main jars, direct and through JavascriptCompiler: 7 of 7 fail with assertions on and off.
• The acosh negative-argument symptom was found by the independent reproducer, who wrote their own tests from the claim only. So the label is semi-formal: the hypothesis came from reading, the solvers bounded it.