Offending token will sometimes be unexpected

[surister]

from cratedb_sqlparse import sqlparse

query = """
SELECT COUNT(*) FOM doc.tbl f WHERE f.id = 1;

INSERT INTO doc.tbl VALUES (1, 23, 4);
"""
t = sqlparse(query)
t1 = t[0]
print(t1.exception.original_query_with_error_marked)

Error message:

SELECT COUNT(*) FOM doc.tbl f WHERE f.id = 1;
                    ^^^

INSERT INTO doc.tbl VALUES (1, 23, 4);

I assume this happens because antlr4 doesn't know that 'FOM' should be a 'FROM' and therefore the error it finds is InputMismatchException[line 2:20 mismatched input 'doc' expecting {<EOF>, ';'}]

The offendingToken that antrl4 gives is:

class ExceptionCollectorListener(ErrorListener):
    def __init__(self):
        self.errors = []

    def syntaxError(self, recognizer, offendingSymbol, line, column, msg, e):
        print(offendingSymbol)

[@10,21:23='doc',<312>,2:20]

[surister]

Related https://stackoverflow.com/questions/76980636/antlr-how-to-include-information-about-the-offending-symbol-in-the-error-messag