Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[dtp-dev] SQLParserCompletionEngine offers too many choice inside LOOKAHEAD
Title: SQLParserCompletionEngine offers too many choice inside LOOKAHEAD

I am trying to provide a parser for Teradata SQL and have encountered times where far too many choices are offered by the SQL editor's content assist.

There is a rule in the Derby and Generic SQL Parsers that exhibits this behavior.  The rule in those parsers is:

    void predicate() : {}
    {
          LOOKAHEAD(<EXISTS>) exists_predicate() //starts with exists
        | LOOKAHEAD( ("(")+ <SELECT> | _expression_() ) (_expression_()
             (
                 LOOKAHEAD( comp_op() <ANY> ) any_predicate()
               | LOOKAHEAD( comp_op() UK_ALL() ) all_predicate()
               | LOOKAHEAD( comp_op() | join_op() ) comparison_predicate()
               | LOOKAHEAD( [ <NOT> ] <BETWEEN> ) between_predicate()
               | LOOKAHEAD( <IS> [ <NOT> ] ) null_predicate()
               | LOOKAHEAD([ <NOT> ]  <IN> ) in_predicate()
               | LOOKAHEAD([ <NOT> ] <LIKE> ) like_predicate()
             )
           )
        | LOOKAHEAD( row_constructor()) row_constructor() row_comparison_predicate()
    }

The problem occurs if your text in a SQL editor whose connection type is either Derby or Generic JDBC is something like:

        SELECT * FROM tbl WHERE x NOT
and you display the list of choices for the next token (by pressing Ctrl-Space or just pausing).  The next valid token is BETWEEN, IN or LIKE.  Those tokens (for some reason) are not appearing in the list but all the choices that would have been displayed had I not included the "NOT" are shown.

After this token anything is accepted and the subsequent lists of choices offered by the content assist are the same.

If my SQL editor contains:
        SELECT * FROM tbl WHERE x NOT BETWEEN
then the content assist gets back on track.

The problem seems to be from a behavior of the parsers generated by JavaCC.  On the web page https://javacc.dev.java.net/doc/errorrecovery.html there is the paragraph:

    There is a method in the generated parser called "generateParseException". You can call this method anytime you wish to generate an object of type ParseException. This object will contain all the choices that the parser has attempted since the last successfully consumed token.

(I added the bold.)  The problem seems to be that the "NOT" has not been consumed by the parser since it is still in a LOOKAHEAD.  So what appears to show up in the list of choices offered by the content assist are the tokens that could have appeared instead of the "NOT" token (not the tokens that could have followed it).

In my Teradata grammar the content assist choices include the tokens that could have appeared instead of the last token and the tokens that could follow it.

Is this a shortcoming in JavaCC-generated parsers?  Or is there something the content assist engine or I can do to have only the correct choices be offered?

--Charles Eutsler

Back to the top