Eclipse Community Forums: TMF (Xtext) » Migrating from ANTLR3 to ANTLR4

Home » Modeling » TMF (Xtext) » Migrating from ANTLR3 to ANTLR4

Migrating from ANTLR3 to ANTLR4 [message #1838808]

Fri, 05 March 2021 06:07

Eclipse User

Hey,

I'm a student at the Technical University of Vienna and was proposed the idea of migrating Xtext from ANTLR v3 to v4 for my Bachelor's Thesis. The main motivation being the left-recursion support that v4 brings.

I've read through some of the community discussions:

https://www.eclipse.org/forums/index.php/t/640630/
https://www.eclipse.org/forums/index.php/t/1084702/
https://github.com/eclipse/xtext/issues/1929

Reading through these threads I would like your input on a couple of points.

1)
As it currently stands there are no plans from you to migrate to v4. Some reasons are:

a) The benefits are not worth the time and effort.
b) There are some concerns regarding performance.
c) We would lose static checking (and therefore the ability to check if the language is ambiguous)

How strong are points b) and c)? Would these points speak against merging this change upstream?

2)
How much time and effort do you think this project would require?
As it stands the Bachelor's Thesis is calculated with about 250 work hours (though many peers fail to hold that limit).

Is 250 hours a reasonable time or do you think this project vastly exceeds the scope of that?

Thanks for your input.

Re: Migrating from ANTLR3 to ANTLR4 [message #1838816 is a reply to message #1838808]

Fri, 05 March 2021 09:47

Eclipse User

Hi

There is a further problem. While the EF encourages all projects to use the latest version in Orbit i.e ANTLR 4, the significant incompatibilities between ANTLR 3 and ANTLR 4 mean that many (? all) projects have elected to stick with ANTLR 3 with unification on the latest ANTLR 3 the limit of evolution.

So if Xtext moves to ANTLR 4 there is at least an inconvenience for projects that continue to use ANTLR 3 and a potentially blocking problem for projects like Papyrus that may find that their re-use of Xtext mandates ANTLR 4 while their re-use of ATL/Epsilon/legacy-Xtext-derivations/... mandates ANTLR 3.

So, again, if Xtext moves to ANTLR 4, there may be significant pressure to move back to ANTLR 3, or for an ANTLR 3 fork of Xtext to be used by existing projects.

If you really want to migrate Xtext, I would regard a migration to LPG as much more useful; smaller and faster [1] and LALR checked. You can find some QVTo transformations that go some way towards this in GIT\org.eclipse.ocl\examples\org.eclipse.ocl.examples.xtext2lpg. NB Xtext already has left recursion in the special case of current=*. My recent work on an anlternative serializer/formatter has made me realize how inddependent of ANTLR Xtext is. An Xtext user has no need to know that there is an ANTLR engine under the hood.

250 hours seems plenty of time for what should be only a modest accommodation of API breakages. Surely ANTLR 4 is not fundamentally different to ANTLR 3?

Regards

Ed Willink

[1] http://www.eclipse.org/modeling/mdt/ocl/docs/publications/MODELS2010OCLWorkshop/oclxtext.pdf

Re: Migrating from ANTLR3 to ANTLR4 [message #1838926 is a reply to message #1838808]

Tue, 09 March 2021 12:13

Eclipse User

Hello Fabian,

thank you for your interest in this ambitious project. I've tried to answer your questions inline.
Quote:

Reading through these threads I would like your input on a couple of points.

1)
As it currently stands there are no plans from you to migrate to v4. Some reasons are:

a) The benefits are not worth the time and effort.
b) There are some concerns regarding performance.
c) We would lose static checking (and therefore the ability to check if the language is ambiguous)

How strong are points b) and c)? Would these points speak against merging this change upstream?

The points 2 + 3 are crucial for the bigger parts of the Xtext user base. Static checks are extremely valuable especially with such a condensed language as the Xtext grammar language. Performance is absolutely important for the bigger clients of the framework. I'll be honest: We are by no means willing to make compromises on (b) or (c).
Regarding (a) effort is not only measured in the initial effort but also in the maintenance effort. The lack of static analysis will likely lead to bigger effort when it comes to bugfixing, analysis of weird feedback or simply understanding what the parser did. This is not only the case for the maintainers for also for the adopters of the framework.

Quote:

2)
How much time and effort do you think this project would require?
As it stands the Bachelor's Thesis is calculated with about 250 work hours (though many peers fail to hold that limit).

Is 250 hours a reasonable time or do you think this project vastly exceeds the scope of that?

Personally I wouldn't expect that I could do this in 250/8 = 40 days = 2 months time.

Quote:

The main motivation being the left-recursion support that v4 brings.

I often hear this argument but in fact left recursion is really not a big problem because the applied technique is always the same. It's code that needs to put once into the grammar and afterwards it just works. In 250hours, your could probably refactor 250 grammars in such way. If that is the pain point which shall be addressed, I'd rather work on a quickfix or quickassist feature along with a validation that projects Antlr analysis feedback into the grammar editor.

Previous Topic:	Ambiguous grammar
Next Topic:	Include * in Xtext grammar

Goto Forum:

-=] Back to Top [=-

Current Time: Sun Jul 06 18:31:23 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter