I have been meaning to ask for this on the Orion list.
Have you considered contributing the tolerant code from your fork
into
esprima?
There seems to be some work happening on esprima towards a 3.0
release
which
seems to introduce some additional tolerant parsing [1]. I think this
would make it
easier for Orion to upgrade to a new version and timing may be right.
I have opened [2][3] to get the conversation started.
[1]
[https://github.com/jquery/esprima/commit/9d5adfd7a872a348f15746697f30326c7ec064fd](https://github.com/jquery/esprima/commit/9d5adfd7a872a348f15746697f30326c7ec064fd)
[2]
[https://github.com/jquery/esprima/issues/1482](https://github.com/jquery/esprima/issues/1482)
[3]
[https://github.com/jquery/esprima/issues/1481](https://github.com/jquery/esprima/issues/1481)
â
Gorkem
On 14 Mar 2016, at 11:26, Michael Rennie wrote:
> In Orion, we have a modified version of Esprima (its a bit old now,
> version 2.0), that has more tolerance in it than the stock Esprima
> does. It might provide the level of recovery you are looking for.
> Â
> Our recovery is done in a few ways:
> Â
> 1. we catch and record all exceptions, to make it far less 'throwy'
> 2. we perform node fill-ins when an exception would leave the AST
is
> bad state
> 3. in some cases we rewind and try again based on state / tokens /
and
> line endings
> Â
> Our version can be found here:Â
> Â
>
[[http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/tree/bundles/org.eclipse.orion.client.javascript/web/esprima/esprima.js](http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/tree/bundles/org.eclipse.orion.client.javascript/web/esprima/esprima.js)]([http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/tree/bundles/org.eclipse.orion.client.javascript/web/esprima/esprima.js](http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/tree/bundles/org.eclipse.orion.client.javascript/web/esprima/esprima.js))
> Â
> The tests we run on it (to give an idea of the kinds of things we
can
> recover from):
> Â
>
[[http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/tree/bundles/org.eclipse.orion.client.javascript/web/js-tests/javascript/esprimaTolerantTests.js](http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/tree/bundles/org.eclipse.orion.client.javascript/web/js-tests/javascript/esprimaTolerantTests.js)]([http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/tree/bundles/org.eclipse.orion.client.javascript/web/js-tests/javascript/esprimaTolerantTests.js](http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/tree/bundles/org.eclipse.orion.client.javascript/web/js-tests/javascript/esprimaTolerantTests.js))
> Â
> And lastly, the bug for us to upgrade to the latest version of
> Esprima:
> Â
>
[[https://bugs.eclipse.org/bugs/show_bug.cgi?id=473765](https://bugs.eclipse.org/bugs/show_bug.cgi?id=473765)]([https://bugs.eclipse.org/bugs/show_bug.cgi?id=473765](https://bugs.eclipse.org/bugs/show_bug.cgi?id=473765))
> Â
> Michael Rennie
> Â
> Â ----- Original message -----
>> From: Eugene Melekhov <emvv@xxxxxxx>
>> Sent by: wtp-dev-bounces@xxxxxxxxxxx
>> To: "General discussion of project-wide or architectural issues."
>> <wtp-dev@xxxxxxxxxxx>
>> Cc:
>> Subject: Re: [wtp-dev] JSDT Parser
>> Date: Sat, Mar 12, 2016 10:31 AM
>> Â
> Angelo,
>>
>> Yes, sure, I used that option. The problem is that esprima
actually
>> throws a lot of syntax exceptions:
>>
>> grep -c throwUnexpectedToken esprima.js
>> 56
>> grep -c tolerateUnexpectedToken esprima.js
>> 29
>>
>> That means that in 2/3 error cases it throws exception and
tolerate
>> only 1/3. Shift meanwhile doesn't throws exceptions
>> in "Early Error" cases like this 0=0, which is tolerated by
esprima.
>> I have a suspicion that most of esprima
>> "tolerated errors" are from that "Early Error" category, which
means
>> that in fact Shift behaves like esprima :-)
>>
>> I thought that it would be interesting to compare shift and
esprima
>> behavior on large enough set of erroneous code
>> fragments. It might happen that they are very close actually :-)
>>
>>
>> > I'm not a big expert with esprima, but do you use tolerant
option
>> like explained
>>
at [[http://esprima.org/doc/](http://esprima.org/doc/)]([http://esprima.org/doc/](http://esprima.org/doc/)) ?
>>
>> > 2016-03-12 10:25 GMT+01:00 Eugene Melekhov <emvv@xxxxxxx>:
>>
>> > Angelo,
>> > Â
>> > Â One more thing about parser tolerance. esprima for example
throws
>> an exception parsing this "relatively simple"
>> > Â declarations:
>> > Â
>> > Â //function foo(a, b {}
>> > Â
>> > Â //function foo(a, b, c,) {}
>> > Â
>> > Â //function foo(a, b, {} {}
>> > Â
>> > Â
>> Â >> Yes sure, but it seems that Shift doesn't support tolerant
>> parser which is very required for a JS editor. See
>>
[[https://github.com/shapesecurity/shift-java/issues/93](https://github.com/shapesecurity/shift-java/issues/93)]([https://github.com/shapesecurity/shift-java/issues/93](https://github.com/shapesecurity/shift-java/issues/93))
>> > Â
>> > Â
>> > Â
>> > --
>> > Â Eugene Melekhov
>> > Â
>> > Â _______________________________________________
>> > Â wtp-dev mailing list
>> > Â wtp-dev@xxxxxxxxxxx
>> > Â To change your delivery options, retrieve your password, or
>> unsubscribe from this list, visit
>> >
>>
 [[https://dev.eclipse.org/mailman/listinfo/wtp-dev](https://dev.eclipse.org/mailman/listinfo/wtp-dev)]([https://dev.eclipse.org/mailman/listinfo/wtp-dev](https://dev.eclipse.org/mailman/listinfo/wtp-dev))
>> > Â
>>
>>
>>
>>
>> --
>> Eugene Melekhov
>>
>> _______________________________________________
>> wtp-dev mailing list
>> wtp-dev@xxxxxxxxxxx
>> To change your delivery options, retrieve your password, or
>> unsubscribe from this list, visit
>>
[[https://dev.eclipse.org/mailman/listinfo/wtp-dev](https://dev.eclipse.org/mailman/listinfo/wtp-dev)]([https://dev.eclipse.org/mailman/listinfo/wtp-dev](https://dev.eclipse.org/mailman/listinfo/wtp-dev))
>
>
> _______________________________________________
> wtp-dev mailing list
> wtp-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or
> unsubscribe from this list, visit
>
[https://dev.eclipse.org/mailman/listinfo/wtp-dev](https://dev.eclipse.org/mailman/listinfo/wtp-dev)
_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
[https://dev.eclipse.org/mailman/listinfo/wtp-dev](https://dev.eclipse.org/mailman/listinfo/wtp-dev)
