SimpleWKTShapeParser rejects MULTIPOINT ((1 2), (3 4)), the standard WKT form with each point in its own parentheses

SimpleWKTShapeParser, the public WKT reader in core org.apache.lucene.geo, says it parses WKT and complies with ISO/IEC 13249-3:2016. In the Simple Features WKT grammar each member of a multipoint is a point text in its own parentheses, MULTIPOINT ((10 10), (20 20)); the bare MULTIPOINT (10 10, 20 20) is the older, widely accepted variant. The parser accepts the bare form and a single parenthesized point, but MULTIPOINT ((1 2), (3 4)) throws java.text.ParseException: expected ) but found: ','. In parseCoordinates the code before the loop reads a parenthesized first point's ( and numbers but not its ); the loop condition then reads that ) as the end of the whole list, and the check after the loop finds the ,. Points read inside the loop get their ) checked at once, so acceptance depends on position: MULTIPOINT (1 2, (3 4), 5 6) parses, MULTIPOINT (1 2, (3 4)) fails with expected word but found: END-OF-STREAM. The failure is loud and never yields a wrong shape. It dates from the class's first commit (LUCENE-8538, 2018) and is in every release from 7.6.0 to 10.6.0 and on main. Reading each multipoint member as bare or ( point ) fixes it.

• SimpleWKTShapeParser: a public class in core org.apache.lucene.geo (exported, no internal or experimental tag) with two entry points, parse(String) and parseExpectedType(String, ShapeType), both declaring a checked java.text.ParseException (SimpleWKTShapeParser.java:50-55 at 645d4fe14a20).
• parseCoordinates (:122-145): the shared helper that reads a comma-separated list of coordinates up to its closing ). Multipoints, lines, polygon rings and holes all use it.
• nextCloserOrComma (:374-383): reads one token and returns it if it is , or ). The loop over the list continues on , and stops on ).

For MULTIPOINT ((1 2), (3 4)), after parseMultiPoint has read the list's ( (:161) and called parseCoordinates (:167):

1. :126: the next token is (, not a number, so nextWord reads it and sets isOpenParen = true; parseCoordinate reads 1 and 2 (:127). The point's ) is not read.
2. :130: nextCloserOrComma reads that ), the first point's closer, so the loop ends at once: the parser now believes the whole list is closed.
3. :141: isOpenParen is still true, so nextCloser reads the next token, ,, and throws ParseException("expected ) but found: ','") (:351-352).

MULTIPOINT ((1 2)) passes by coincidence: the point's ) ends the loop and the list's ) satisfies the check. A parenthesized point read inside the loop has its ) checked at once (:135-138), and isOpenParen is reset each iteration (:131), so a mid-list one works. A parenthesized last point leaves isOpenParen set after the list's ) was read, so :141 asks for one more ) and hits the end of input: expected word but found: END-OF-STREAM (:294).

Polygon holes rely on the first-position branch: parsePolygonHole (:203-210) calls parseCoordinates with the hole's own ( unread, :126 swallows it, and :131 clears the flag at the hole's second coordinate. With the check after the loop, holes still parse; a fix that checks the first point's ) early must therefore also make parsePolygonHole read its own (.

Side note: the helper's own messages "expected: [)] but found: [...]" (:136-137, :142-143) are never produced, because nextCloser throws first for anything other than ) (:347-353).

• Who is hit. Callers of SimpleWKTShapeParser.parse or parseExpectedType (public static, checked ParseException) with a standard-form multipoint of two or more points, alone or inside a GEOMETRYCOLLECTION. The class is in an exported core package (module-info.java:38) with no internal or experimental tag. In the Lucene tree only tests call it (TestSimpleWKTShapeParsing, TestTessellator, TestShapeDocValues).
• Downstream users (read only, nothing run): rdf4j's Lucene index catches the ParseException, logs a warning and adds no geo fields, so such a geometry is left out of its geo index and geo queries miss that record; a copy of that class lives in INCEpTION; Anserini's generator does not handle MULTIPOINT anyway. No use was found in Elasticsearch, OpenSearch or Solr (code search, not exhaustive).
• Who writes the standard form. PostGIS's ISO WKT writer since PR #682 ("MultiPoint WKT writer: use parentheses in sub-members in WKT_ISO format", merged 2022-04-12). Elasticsearch #151876 (open) says JTS writes MULTIPOINT ((0 0),(1 1), ...) and that it "can happen in the wild"; Elasticsearch #138162 (open) asks for the same form.
• Default, no setting. The parser has no options; assertions on or off make no difference.
• What happens. The first such multipoint is refused with a checked ParseException: no wrong or partial shape, no hang (a 200,000-point standard-form input fails in 0.2 ms, at the first ,), nothing written.
• Scale and likelihood. Two points are enough. Multipoints are a minority geometry type and no user report turned up in the recorded searches, so it is rated plausible, low priority.
• Workaround. Rewrite to the bare form, MULTIPOINT (1 2, 3 4), before parsing.

Pure string parsing: no index, no randomness (the seed does not matter). The ParseException is rethrown as an AssertionError that states expected and actual, with the original exception as its cause. It compiles against 9.1.0 and later; before 9.1.0 (7.x, 8.x, 9.0.0) import org.apache.lucene.util.LuceneTestCase instead, and before 8.5.0 add the sandbox jar, where the class lived then.

As the regression test: the same test, for example as a new method in TestSimpleWKTShapeParsing.

cp $REPRO/TestWKTMultiPointParenthesizedPoints.java $LUCENE/lucene/core/src/test/org/apache/lucene/geo/
cd $LUCENE && ./gradlew :lucene:core:test --tests org.apache.lucene.geo.TestWKTMultiPointParenthesizedPoints -Ptests.seed=DEADBEEF
# assertions off: add -Ptests.asserts=false

$ ./gradlew :lucene:core:test --tests org.apache.lucene.geo.TestWKTMultiPointParenthesizedPoints -Ptests.seed=DEADBEEF

TestWKTMultiPointParenthesizedPoints > testTwoParenthesizedPoints FAILED
    java.lang.AssertionError: expected MULTIPOINT ((1 2), (3 4)) to parse as {{1, 2}, {3, 4}}, got java.text.ParseException: expected ) but found: ','
        at __randomizedtesting.SeedInfo.seed([DEADBEEF:76696EB2AFA5FE2B]:0)
        at org.apache.lucene.geo.TestWKTMultiPointParenthesizedPoints.testTwoParenthesizedPoints(TestWKTMultiPointParenthesizedPoints.java:30)

        Caused by:
        java.text.ParseException: expected ) but found: ','
            at org.apache.lucene.geo.SimpleWKTShapeParser.nextCloser(SimpleWKTShapeParser.java:352)
            at org.apache.lucene.geo.SimpleWKTShapeParser.parseCoordinates(SimpleWKTShapeParser.java:141)
            at org.apache.lucene.geo.SimpleWKTShapeParser.parseMultiPoint(SimpleWKTShapeParser.java:167)
            at org.apache.lucene.geo.SimpleWKTShapeParser.parseGeometry(SimpleWKTShapeParser.java:90)
            at org.apache.lucene.geo.SimpleWKTShapeParser.parseExpectedType(SimpleWKTShapeParser.java:69)
            at org.apache.lucene.geo.SimpleWKTShapeParser.parse(SimpleWKTShapeParser.java:51)
            at org.apache.lucene.geo.TestWKTMultiPointParenthesizedPoints.testTwoParenthesizedPoints(TestWKTMultiPointParenthesizedPoints.java:28)

ERROR: 1 test has failed:

  - org.apache.lucene.geo.TestWKTMultiPointParenthesizedPoints.testTwoParenthesizedPoints (:lucene:core)
BUILD FAILED

• Contract. Class javadoc (SimpleWKTShapeParser.java:29-34): "Parses shape geometry represented in WKT format", "complies with OGC® document: 12-063r5 and ISO/IEC 13249-3:2016 standard". The Simple Features grammar, read through the OGC's standalone WKT reference and secondary sources (the ISO text itself was not read), defines <multipoint text> as ( <point text> {, <point text>}* ) with <point text> ::= ( <point> ), example MULTIPOINT ((10 10), (20 20)); one transcription marks the bare form "non-standard but widely used". 12-063r5 is the WKT standard for coordinate reference systems, not geometry, so it is not the argument here.
• Code intent. Dedicated ( branches (:126, :132) and ) checks (:135-138, :141-144); acceptance by position (one point yes, mid-list yes, first or last no) is not described as intended in any documentation or commit message found.
• Origin. The class is a port of Elasticsearch's GeoWKTParser (same identifiers, same javadoc line). In Elasticsearch v6.4.0, parseCoordinateList checks the first point's ) right after it, before the loop, and reads polygon holes through a separate opener; the port checks it after the loop, which keeps parsePolygonHole working and breaks multipoints. Elasticsearch PR #27417 (2017), the ported code, lists MULTIPOINT ((10 40), (40 30), ...) as supported. The Lucene commit gives no reason.
• Issue, docs, tests. #9584 (LUCENE-8538) lists MULTIPOINT -> double[][] with no syntax limit; CHANGES.txt (7.6.0, :7093-7094) reads "Add a Simple WKT Shape Parser for creating Lucene Geometries (Polygon, Line, Rectangle) from WKT format", naming those three types and no syntax limit; no javadoc, CHANGES.txt or MIGRATE.md entry rules the form out. TestSimpleWKTShapeParsing.testMultiPoint (:45-60) tests only the bare form. In-tree precedent (as the cross-model reviewer pointed out): TestGeo3d (:42) gives MULTIPOINT ((10 40), (40 30), (20 20), (30 10)) to spatial4j's WKT reader and expects it to parse.

Strongest case for intended: the class is "Simple"; the introducing issue calls it "useful for simple tests, but also helps for benchmarking from real world data"; it has no in-tree production caller; it supports only a WKT subset (no Z/M tags, no TIN); its compliance claim links a CRS standard; its one MULTIPOINT test uses the bare form; and it fails loudly. So the parenthesized form would be an enhancement.

Why it fails: that covers features the parser never tried to support. Here it does try: explicit branches for parenthesized points that work for one point and in mid-list and fail only at the ends of a list, while the original it was ported from handles the form and the API is public and untagged. It limits severity (low priority), not validity.

Two options were run through gradle in a scratch worktree of 645d4fe14a20; B is recommended because it changes nothing outside MULTIPOINT. Moving only the first point's ) check before the loop, as in the Elasticsearch original, would break every polygon hole unless parsePolygonHole also reads its own (, which is option A.

In SimpleWKTShapeParser.parseMultiPoint (:159-173), replace the call to the shared parseCoordinates (:167) with its own loop.

     ArrayList<Double> lats = new ArrayList<>();
     ArrayList<Double> lons = new ArrayList<>();
-    parseCoordinates(stream, lats, lons);
+    do {
+      // each point is written either bare, "1 2", or in its own parentheses, "(1 2)"
+      if (isNumberNext(stream)) {
+        parseCoordinate(stream, lats, lons);
+      } else {
+        nextOpener(stream);
+        parseCoordinate(stream, lats, lons);
+        nextCloser(stream);
+      }
+    } while (nextCloserOrComma(stream).equals(COMMA));

Trade-offs

• Accepts the standard form, the bare form and mixed lists (which the current in-loop branch already means to accept), with 3D points and inside collections.
• LINESTRING, polygon and hole parsing are unchanged (checked: LINESTRING (1 2, (3 4), 5 6) still parses, LINESTRING ((1 2), (3 4)) is still refused; neither is standard WKT).
• Rejects one malformed multipoint that main accepts, MULTIPOINT ((1 2, 3 4) (unbalanced; main reads two points), with expected ) but found: ','.
• Leaves the shared helper's quirk in place for other types. Still one linear pass over the tokens.
• Covers non-empty point texts only: point-level EMPTY members, legal in the grammar (MULTIPOINT ((1 2), EMPTY), MULTIPOINT (EMPTY)), are rejected on main and with A or B alike (invalid number found: EMPTY, because isNumberNext treats any word as a number); supporting them is a separate decision.

What the checks validated

Gradle, scratch worktree of 645d4fe14a20 (2026-11-01): the repro fails on the stock tree and passes with B, with assertions on and off; all 17 org.apache.lucene.geo test classes plus TestShapeDocValues pass (200 tests, 0 failures, 7 skipped, incl. TestSimpleWKTShapeParsing and TestTessellator); :lucene:core:check -x test passes. With B shadowed over the main jars, all 104,760 well-formed inputs of a 200,000-input differential probe parse to exactly the written shape.

parseCoordinates reads every coordinate through a new parseBareOrParenthesizedCoordinate (bare, or nextOpener + coordinate + nextCloser), and parsePolygonHole reads the hole's own ( first (the finder's sketch; +14/-16 lines).

Trade-offs

• One rule for every coordinate list; holes no longer rely on the quirk.
• Changes other types: LINESTRING ((1 2), (3 4)) is then accepted (not standard WKT), and malformed strings main accepts are refused, such as LINESTRING ((1 2, 3 4) and a polygon hole without its own ( (expected ( but found: 2). In the study's 200,000-input probe, accepted inputs with unbalanced parentheses went from 1,277-1,338 per run to 0.
• Stricter is arguably better, but it is a behaviour change outside MULTIPOINT for maintainers to decide on.

What the checks validated

Gradle, same worktree: the repro passes with A, with assertions on and off; the same 18 classes pass (200 tests, 0 failures, 7 skipped). Earlier, with the sketch shadowed: the hunt tests pass and the differential probe finds no well-formed input rejected or misread.

Method: non-formal (a differential probe, then executed tests). On 2026-10-26 the hunt unit for the LatLon geo queries used, as its secondary method, a probe that parses every WKT geometry type the class supports, including EMPTY, ENVELOPE/BBOX, 3D coordinates and collections, and compares the result with the shape as written. MULTIPOINT ((1 2), (3 4)) was rejected while MULTIPOINT ((1 2)) parsed; following parseCoordinates token by token gave the root cause. A 4-test public-API JUnit (2 controls, 2 bug cases) reproduced it on main (it was later run on 11 releases from 7.6.0 to 10.5.2, 9 by the history step and 2 by the finder); independent checkers then reproduced it with their own tests, and it was cut to the one-test repro.