Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » Modeling (top-level project) » Textual Modelling Framework?
Textual Modelling Framework? [message #376408] Tue, 23 May 2006 15:05 Go to next message
Sven Efftinge is currently offline Sven Efftinge
Messages: 1769
Registered: July 2009
Senior Member
Hi,

I just met Artem at the JAX conference in Germany, where we talked a bit
about different approaches for textual notations for DSLs.

I wanted to point you to the xText framework we are currently developing
at openArchitectureWare.
It offers (generates) a textual parser and a textual eclipse editor
based on a grammar described in a simplified BNF-notation.

The generated parser is based on AntLR and creates a (dynamic) EMF
model. The generated text editor provides syntax highlighting, syntax
checking, constraint checking and an outline view.

I've written a blog entry showing some more details (and screenshots):
http://effi-blog.blogspot.com/2006/04/textual-dsl-framework- xtext.html

The xText framework was also a part of our session at "Eclipse Forum
Europe", which shows the integration of EMF and GMF with oAW (and
xText). Some of you may find that interesting, too.
The slides can be found here:
http://www.eclipse.org/gmt/oaw/doc/s10_gmfAndOAW_slides.pdf

We are very interested in sharing ideas about textual DSL design and
would be happy if we could contribute some ideas or code to a possible
TMF (Textual Modelling Framework) if there are plans to create such a
project.


best regards,
Sven


--
Need professional support on Xtext or Xtend?
Mail to: xtext (at) itemis.com
Twitter : @svenefftinge
Blog : blog.efftinge.de
Re: Textual Modelling Framework? [message #376409 is a reply to message #376408] Wed, 24 May 2006 06:55 Go to previous messageGo to next message
Richard Gronback is currently offline Richard Gronback
Messages: 23
Registered: July 2009
Junior Member
Hi Sven,

I recently came across xText while reading this
https://bugs.eclipse.org/bugs/show_bug.cgi?id=141679
Re: Textual Modelling Framework? [message #376410 is a reply to message #376409] Wed, 24 May 2006 09:37 Go to previous messageGo to next message
Richard Gronback is currently offline Richard Gronback
Messages: 605
Registered: July 2009
Senior Member
I guess the remainder of my note didn't make it...

What it said was that you might consider getting together with Artem and
Chris Daly (Emfatic) to work on a project proposal. It's definitely a project
we'd like to see started, as it's on our charter. We look forward to your
contribution :)

Best Regards,
Rich
Re: Textual Modelling Framework? [message #376411 is a reply to message #376410] Wed, 24 May 2006 11:01 Go to previous messageGo to next message
Ed Merks is currently offline Ed Merks
Messages: 26152
Registered: July 2009
Senior Member
Rich,

I was asking Chris for a comparison between this work and his Gymnast
work; there are lots of similarities and a few differences.
Unfortunately, because of his use of ANTLR, Chris' donation is being
held up. I suspect that Sven's work will run into the same legal
barrier. :-( Other projects (OCL) that used ANTLR have switched to
LPG. I'll follow up with Janet to confirm if there is now an absolute
barrier against the use of ANTLR or if perhaps the development work to
migrate to a framework with a more sound pedigree can take place within
the Eclipse open source community rather than being forced outside of it.


Richard Gronback wrote:

> I guess the remainder of my note didn't make it...
>
> What it said was that you might consider getting together with Artem
> and Chris Daly (Emfatic) to work on a project proposal. It's
> definitely a project we'd like to see started, as it's on our
> charter. We look forward to your contribution :)
>
> Best Regards,
> Rich
>
>
related idea at Google Summer of Code [message #376412 is a reply to message #376408] Wed, 24 May 2006 11:22 Go to previous messageGo to next message
Miguel Garcia is currently offline Miguel Garcia
Messages: 40
Registered: July 2009
Member
The following project idea

> Eclipse IDE generator: building on some previous research work (Chris
> Laffra), provide an Eclipse IDE generation environment derived from a
> language grammar. This project would allow the creation of basic support
> of new or existing languages in Eclipse rapidly.

appears listed at
http://wiki.eclipse.org/index.php/Google_Summer_of_Code_2006

Given that applicants were to be notified before May 22nd, 2006 the outcome
should be out already.


Miguel Garcia
Re: Textual Modelling Framework? [message #376413 is a reply to message #376411] Wed, 24 May 2006 11:41 Go to previous messageGo to next message
Srgjan Srepfler is currently offline Srgjan Srepfler
Messages: 33
Registered: July 2009
Member
Ed Merks wrote:
> Rich,
>
> I was asking Chris for a comparison between this work and his Gymnast
> work; there are lots of similarities and a few differences.
> Unfortunately, because of his use of ANTLR, Chris' donation is being
> held up. I suspect that Sven's work will run into the same legal
> barrier. :-( Other projects (OCL) that used ANTLR have switched to
> LPG. I'll follow up with Janet to confirm if there is now an absolute
> barrier against the use of ANTLR or if perhaps the development work to
> migrate to a framework with a more sound pedigree can take place within
> the Eclipse open source community rather than being forced outside of it.
>
>
> Richard Gronback wrote:
>
>> I guess the remainder of my note didn't make it...
>>
>> What it said was that you might consider getting together with Artem
>> and Chris Daly (Emfatic) to work on a project proposal. It's
>> definitely a project we'd like to see started, as it's on our
>> charter. We look forward to your contribution :)
>>
>> Best Regards,
>> Rich
>>
>>
Is ANTLR inferior to LPG or is it a licensing issue?
Srgjan
Re: Textual Modelling Framework? [message #376414 is a reply to message #376413] Wed, 24 May 2006 11:52 Go to previous messageGo to next message
Ed Merks is currently offline Ed Merks
Messages: 26152
Registered: July 2009
Senior Member
Srgjan,

The issue is a legal one in that ANTLR has a poorly tracked pedigree.
I.e., it's not entirely clear where everything in ANTLR came from and
whether all contributors intended their work to be licensed as it is.


Srgjan Srepfler wrote:

> Ed Merks wrote:
>
>> Rich,
>>
>> I was asking Chris for a comparison between this work and his Gymnast
>> work; there are lots of similarities and a few differences.
>> Unfortunately, because of his use of ANTLR, Chris' donation is being
>> held up. I suspect that Sven's work will run into the same legal
>> barrier. :-( Other projects (OCL) that used ANTLR have switched to
>> LPG. I'll follow up with Janet to confirm if there is now an
>> absolute barrier against the use of ANTLR or if perhaps the
>> development work to migrate to a framework with a more sound pedigree
>> can take place within the Eclipse open source community rather than
>> being forced outside of it.
>>
>>
>> Richard Gronback wrote:
>>
>>> I guess the remainder of my note didn't make it...
>>>
>>> What it said was that you might consider getting together with Artem
>>> and Chris Daly (Emfatic) to work on a project proposal. It's
>>> definitely a project we'd like to see started, as it's on our
>>> charter. We look forward to your contribution :)
>>>
>>> Best Regards,
>>> Rich
>>>
>>>
> Is ANTLR inferior to LPG or is it a licensing issue?
> Srgjan
Re: Textual Modelling Framework? [message #376415 is a reply to message #376408] Wed, 24 May 2006 12:47 Go to previous messageGo to next message
Constantine Plotnikov is currently offline Constantine Plotnikov
Messages: 45
Registered: July 2009
Member
Hi,

I have the project in the same direction. It is ETL (see
http://etl.sourceforge.net/etl-java-0_2_0/doc/readme.html for online
documentation and
http://sourceforge.net/project/showfiles.php?group_id=153075 for demos
and documentation).

ETL also allows creating a EMF tree out of text. It is also possible to
create other kinds of trees using the framework (like using plain
JavaBeans) because ETL provides a generic pull parser. Note that ETL
does not requires Java code generation to create parser from grammar
definition. The grammar is compiled into state machine in runtime. ETL
parser framework component organization is quite similar to one of
typical XML parser.

However in my project I have focused on extensibility and reusability of
textual notation definitions rather than on generality of syntax
definition. As result, in ETL is it is possible to define smaller set of
surface syntaxes than in xText. Deep in grammar definition pipeline, the
grammar is translated to LL(1) form. Additional restrictions come from
shared phrase parser and lexer.

But on other hand it is possible to combine these languages together and
to create new languages from old ones. It is also much simpler to create
definitions of operators. See documentation for examples. And I believe
for every language definable with LL(k) grammar it is possible to define
a language with ETL with the same semantics and which still have a
reasonable notation.

As I understand the extensibility in xText is more limited as it uses
lower-level, but more general constructs to define languages. While
wider range of languages could be defined using it, I'm concerned that
these languages will become things in themselves. Lack of reuse would
not hurt on the small projects. But in larger ones it could be somewhat
counterproductive, as do-not-repeat-yourself principle will be violated.
Another problem is that lack of reuse (except using "copy/paste" reuse
methodology) will lead to languages that differ in minor details like
numeric literals, control constructs, etc. even in DSLs created by
single organization. It will impossible to select a common sub-language
and directly express it as such using grammar definition. This will be
barrier for learning and use of DSLs.

In addition to tree based parsers ETL support pull parsing to support
interpreter read-eval-print loops. I also think that pull parsing will
be more convenient for text editors.

Note that ETL is in much more early development stages than xText. I am
not able to dedicate much time to it as it is still a personal project.
There is no generic customizable text editor for the language yet. It is
believed that such text editor is possible.

I'm also interested if the xText editor supports reparsing of the source
code fragments or always parses entire source.

There might be some integration directions between ETL and xText:
- If editor framework is flexible enough, it can be used to create
general customizable ETL text editor.
- Transformation and constraint component might be reused if they mostly
depend on EMF.
- ETL might be used to define some languages used in xText framework. It
is possible that syntax for sources provided in presentation could be
expressed.
- A specialized xText ETL framework could be created that simplifies
xText editor creation for ETL-based languages.

Constantine


Sven Efftinge wrote:
> Hi,
>
> I just met Artem at the JAX conference in Germany, where we talked a bit
> about different approaches for textual notations for DSLs.
>
> I wanted to point you to the xText framework we are currently developing
> at openArchitectureWare.
> It offers (generates) a textual parser and a textual eclipse editor
> based on a grammar described in a simplified BNF-notation.
>
> The generated parser is based on AntLR and creates a (dynamic) EMF
> model. The generated text editor provides syntax highlighting, syntax
> checking, constraint checking and an outline view.
>
> I've written a blog entry showing some more details (and screenshots):
> http://effi-blog.blogspot.com/2006/04/textual-dsl-framework- xtext.html
>
> The xText framework was also a part of our session at "Eclipse Forum
> Europe", which shows the integration of EMF and GMF with oAW (and
> xText). Some of you may find that interesting, too.
> The slides can be found here:
> http://www.eclipse.org/gmt/oaw/doc/s10_gmfAndOAW_slides.pdf
>
> We are very interested in sharing ideas about textual DSL design and
> would be happy if we could contribute some ideas or code to a possible
> TMF (Textual Modelling Framework) if there are plans to create such a
> project.
>
>
> best regards,
> Sven
Re: Textual Modelling Framework? [message #376416 is a reply to message #376415] Wed, 24 May 2006 15:15 Go to previous messageGo to next message
Sven Efftinge is currently offline Sven Efftinge
Messages: 1769
Registered: July 2009
Senior Member
Hi Constantine,

there are definitely a lot of interesting ideas in your work.
I think the main difference between our approaches is the way of
abstraction.
With xText grammar language one describes a textual language, which is
mapped on an Ecore model. The abstraction lays in hiding all the little
tweeking options, and stuff. So the grammars are very simple and readable.

ETL seems to provide concepts for general purpose programming languages
(with expressions, statements, etc.) and the language descriptions are
not that readable (IMO and of course the provided examples are more
complex) Additionally the language seems to be far more complex (indeed
far more powerful, too)

It's not the goal of xText to define complex languages containing
expressions and stuff, but to be able to write small structural
languages (little languages) in about 5 min (this is what my DSLs are in
about 98% of all cases).
Additionally it is important (IMO), to define the conrete syntax,
because the language should be represented the way the user (domain
expert, who ever that is) wants it to be. I don't want to automatically
have semicolons and curly brackets everywhere.

I really like the extension mechanism ETL provides (I missed such things
in common parser generators). I want to add the possibility of grammar
imports and external rule references to xText so this might relativize
the missing extensibility you mentioned. Of course it's a different
concept and not that flexible and more invasive.

The xText editor uses the generated antLR parser, which is not
incremental. IMO this is not necessary, because the parser is fast enough.

The transformation and constraint component from xText are the languages
"xtend" respective "check" from openArchitectureWare. they are not EMF
dependent but can work on EMF models. BTW.: Both languages use the same
expression sub-language which is another proof for your extensibility
requirement :-)

regards,
Sven

Constantine Plotnikov schrieb:
> Hi,
>
> I have the project in the same direction. It is ETL (see
> http://etl.sourceforge.net/etl-java-0_2_0/doc/readme.html for online
> documentation and
> http://sourceforge.net/project/showfiles.php?group_id=153075 for demos
> and documentation).
>
> ETL also allows creating a EMF tree out of text. It is also possible to
> create other kinds of trees using the framework (like using plain
> JavaBeans) because ETL provides a generic pull parser. Note that ETL
> does not requires Java code generation to create parser from grammar
> definition. The grammar is compiled into state machine in runtime. ETL
> parser framework component organization is quite similar to one of
> typical XML parser.
>
> However in my project I have focused on extensibility and reusability of
> textual notation definitions rather than on generality of syntax
> definition. As result, in ETL is it is possible to define smaller set of
> surface syntaxes than in xText. Deep in grammar definition pipeline, the
> grammar is translated to LL(1) form. Additional restrictions come from
> shared phrase parser and lexer.
>
> But on other hand it is possible to combine these languages together and
> to create new languages from old ones. It is also much simpler to create
> definitions of operators. See documentation for examples. And I believe
> for every language definable with LL(k) grammar it is possible to define
> a language with ETL with the same semantics and which still have a
> reasonable notation.
>
> As I understand the extensibility in xText is more limited as it uses
> lower-level, but more general constructs to define languages. While
> wider range of languages could be defined using it, I'm concerned that
> these languages will become things in themselves. Lack of reuse would
> not hurt on the small projects. But in larger ones it could be somewhat
> counterproductive, as do-not-repeat-yourself principle will be violated.
> Another problem is that lack of reuse (except using "copy/paste" reuse
> methodology) will lead to languages that differ in minor details like
> numeric literals, control constructs, etc. even in DSLs created by
> single organization. It will impossible to select a common sub-language
> and directly express it as such using grammar definition. This will be
> barrier for learning and use of DSLs.
>
> In addition to tree based parsers ETL support pull parsing to support
> interpreter read-eval-print loops. I also think that pull parsing will
> be more convenient for text editors.
>
> Note that ETL is in much more early development stages than xText. I am
> not able to dedicate much time to it as it is still a personal project.
> There is no generic customizable text editor for the language yet. It is
> believed that such text editor is possible.
>
> I'm also interested if the xText editor supports reparsing of the source
> code fragments or always parses entire source.
>
> There might be some integration directions between ETL and xText:
> - If editor framework is flexible enough, it can be used to create
> general customizable ETL text editor.
> - Transformation and constraint component might be reused if they mostly
> depend on EMF.
> - ETL might be used to define some languages used in xText framework. It
> is possible that syntax for sources provided in presentation could be
> expressed.
> - A specialized xText ETL framework could be created that simplifies
> xText editor creation for ETL-based languages.
>
> Constantine
>
>
> Sven Efftinge wrote:
>> Hi,
>>
>> I just met Artem at the JAX conference in Germany, where we talked a
>> bit about different approaches for textual notations for DSLs.
>>
>> I wanted to point you to the xText framework we are currently
>> developing at openArchitectureWare.
>> It offers (generates) a textual parser and a textual eclipse editor
>> based on a grammar described in a simplified BNF-notation.
>>
>> The generated parser is based on AntLR and creates a (dynamic) EMF
>> model. The generated text editor provides syntax highlighting, syntax
>> checking, constraint checking and an outline view.
>>
>> I've written a blog entry showing some more details (and screenshots):
>> http://effi-blog.blogspot.com/2006/04/textual-dsl-framework- xtext.html
>>
>> The xText framework was also a part of our session at "Eclipse Forum
>> Europe", which shows the integration of EMF and GMF with oAW (and
>> xText). Some of you may find that interesting, too.
>> The slides can be found here:
>> http://www.eclipse.org/gmt/oaw/doc/s10_gmfAndOAW_slides.pdf
>>
>> We are very interested in sharing ideas about textual DSL design and
>> would be happy if we could contribute some ideas or code to a possible
>> TMF (Textual Modelling Framework) if there are plans to create such a
>> project.
>>
>>
>> best regards,
>> Sven


--
Need professional support on Xtext or Xtend?
Mail to: xtext (at) itemis.com
Twitter : @svenefftinge
Blog : blog.efftinge.de
Re: Textual Modelling Framework? [message #376417 is a reply to message #376411] Wed, 24 May 2006 15:26 Go to previous messageGo to next message
Sven Efftinge is currently offline Sven Efftinge
Messages: 1769
Registered: July 2009
Senior Member
I also had a look at some presentation slides about Gymnast and indeed
the approaches are very similar.
regarding the license issue:
I thought about switching to Antlr 3 (when it is in beta).
It will be released under BSD.
I don't know LPG but I will have a look at it...

Sven

Ed Merks schrieb:
> Rich,
>
> I was asking Chris for a comparison between this work and his Gymnast
> work; there are lots of similarities and a few differences.
> Unfortunately, because of his use of ANTLR, Chris' donation is being
> held up. I suspect that Sven's work will run into the same legal
> barrier. :-( Other projects (OCL) that used ANTLR have switched to
> LPG. I'll follow up with Janet to confirm if there is now an absolute
> barrier against the use of ANTLR or if perhaps the development work to
> migrate to a framework with a more sound pedigree can take place within
> the Eclipse open source community rather than being forced outside of it.
>
>
> Richard Gronback wrote:
>
>> I guess the remainder of my note didn't make it...
>>
>> What it said was that you might consider getting together with Artem
>> and Chris Daly (Emfatic) to work on a project proposal. It's
>> definitely a project we'd like to see started, as it's on our
>> charter. We look forward to your contribution :)
>>
>> Best Regards,
>> Rich
>>
>>


--
Need professional support on Xtext or Xtend?
Mail to: xtext (at) itemis.com
Twitter : @svenefftinge
Blog : blog.efftinge.de
Re: Textual Modelling Framework? [message #376418 is a reply to message #376417] Wed, 24 May 2006 18:15 Go to previous messageGo to next message
Ed Merks is currently offline Ed Merks
Messages: 26152
Registered: July 2009
Senior Member
This is a multi-part message in MIME format.
--------------010905010501070005010905
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Sven,

This is what Janet Campbell had to say:

To date I have not approved ANTLR for use in any Eclipse project.
Contributions are reviewed and approved/declined based on the
results of the investigation of the code. In the case of ANTLR
pedigree concerns were identified and use of ANTLR in Eclipse was
not approved. A revised version of ANTLR with a cleaner IP trail is
due for release this summer at which time we can revisit the issue.
Please see: http://www.antlr.org/v3/index.html.

LPG has definitely been approved; whether the next version of ANTLR will
be approved will depend on the quality of the legal audit trail for all
its contributions...


Sven Efftinge wrote:

> I also had a look at some presentation slides about Gymnast and indeed
> the approaches are very similar.
> regarding the license issue:
> I thought about switching to Antlr 3 (when it is in beta).
> It will be released under BSD.
> I don't know LPG but I will have a look at it...
>
> Sven
>
> Ed Merks schrieb:
>
>> Rich,
>>
>> I was asking Chris for a comparison between this work and his Gymnast
>> work; there are lots of similarities and a few differences.
>> Unfortunately, because of his use of ANTLR, Chris' donation is being
>> held up. I suspect that Sven's work will run into the same legal
>> barrier. :-( Other projects (OCL) that used ANTLR have switched to
>> LPG. I'll follow up with Janet to confirm if there is now an
>> absolute barrier against the use of ANTLR or if perhaps the
>> development work to migrate to a framework with a more sound pedigree
>> can take place within the Eclipse open source community rather than
>> being forced outside of it.
>>
>>
>> Richard Gronback wrote:
>>
>>> I guess the remainder of my note didn't make it...
>>>
>>> What it said was that you might consider getting together with Artem
>>> and Chris Daly (Emfatic) to work on a project proposal. It's
>>> definitely a project we'd like to see started, as it's on our
>>> charter. We look forward to your contribution :)
>>>
>>> Best Regards,
>>> Rich
>>>
>>>


--------------010905010501070005010905
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Sven,<br>
<br>
This is what Janet Campbell had to say: <br>
<blockquote>To date I have not approved ANTLR for use in any Eclipse
project.&nbsp; Contributions are reviewed and approved/declined based on the
results of the investigation of the code.&nbsp; In the case of ANTLR
pedigree concerns were identified and use of ANTLR in Eclipse was not
approved.&nbsp; A revised version of ANTLR with a cleaner IP trail is due
for release this summer at which time we can revisit the issue.&nbsp; Please
see:&nbsp; <a class="moz-txt-link-freetext" href="http://www.antlr.org/v3/index.html">http://www.antlr.org/v3/index.html</a>. <br>
</blockquote>
LPG has definitely been approved; whether the next version of ANTLR
will be approved will depend on the quality of the legal audit trail
for all its contributions...<br>
<br>
<br>
Sven Efftinge wrote:
<blockquote cite="mide51tul$a1g$1@utils.eclipse.org" type="cite">I also
had a look at some presentation slides about Gymnast and indeed the
approaches are very similar.
<br>
regarding the license issue:
<br>
I thought about switching to Antlr 3 (when it is in beta).
<br>
It will be released under BSD.
<br>
I don't know LPG but I will have a look at it...
<br>
<br>
Sven
<br>
<br>
Ed Merks schrieb:
<br>
<blockquote type="cite">Rich,
<br>
<br>
I was asking Chris for a comparison between this work and his Gymnast
work; there are lots of similarities and a few differences.&nbsp;
Unfortunately, because of his use of ANTLR, Chris' donation is being
held up.&nbsp; I suspect that Sven's work will run into the same legal
barrier.&nbsp; :-(&nbsp; Other projects (OCL) that used ANTLR have switched to
LPG.&nbsp; I'll follow up with Janet to confirm if there is now an absolute
barrier against the use of ANTLR or if perhaps the development work to
migrate to a framework with a more sound pedigree can take place within
the Eclipse open source community rather than being forced outside of
it.
<br>
<br>
<br>
Richard Gronback wrote:
<br>
<br>
<blockquote type="cite">I guess the remainder of my note didn't
make it...
<br>
<br>
What it said was that you might consider getting together with Artem
and Chris Daly (Emfatic) to work on a project proposal.&nbsp; It's
definitely a project we'd like to see started, as it's on our charter.&nbsp;
We look forward to your contribution :)
<br>
<br>
Best Regards,
<br>
Rich
<br>
<br>
<br>
</blockquote>
</blockquote>
</blockquote>
<br>
</body>
</html>

--------------010905010501070005010905--
Re: Textual Modelling Framework? [message #377134 is a reply to message #376416] Thu, 25 May 2006 15:16 Go to previous messageGo to next message
Constantine Plotnikov is currently offline Constantine Plotnikov
Messages: 45
Registered: July 2009
Member
Hi Sven,

The ETL grammar definition language also specifies mapping from source
to AST (that might happen to be EMF/MOF). The simpler grammar definition
language is possible. I will evaluate it for 0.3 after more languages
are created and I will start seeing construct usage patterns better. The
current grammar language lacks any shortcuts and it should definitely
evolve. Theoretically statement definition language can be of about the
same complexity as production language in xText.

Some things in the language that increase its verbosity are done to
support extensibility, but they are possibly too straightforward. For
example name of statement/operator and name of AST object are different
to support extensibility (in xText they are the same as I understand).
This also had allowed defining different mapping to single AST node from
different contexts. I guess it could have been done in other way using
“redefine” declaration but that would have put some restrictions on
extensibility.

Also there is a bit less pressure on verbosity in ETL than in xText. If
language so big that verbosity is being felt, it is likely that some
reuse is possible (common expression language, control constructs, etc.).

The state machine is actually a good sample to demonstrate a need for
extensibility. Initial example is very small and neat. But how long will
it stay as such when you will start receiving user feedback? Feedback
will be likely chained like in the following scenario (guesswork from
experience):

1. The first thing the users will likely want would be guards on
transitions. This will lead to expression sublanguage. Possibly an
existing one like Java or OCL or directly mappable to them.

2. Then it is very likely that users will want action sublanguage that
shares some things with expression sublanguage. Specifying a method name
is a bit too limiting. User will prefer to specify code in place rather
than in some separate template. And the user will prefer to reference
external code libraries directly from action language.

3. Then it is very likely the project with completely different actions
and expression will want to reuse state machine definition but to use
own action and expression sublanguage. This will require factoring out
abstract reusable state machine language (unless copy/paste reuse is
undertaken).

4. Then in the action or expression sublanguage they will likely want to
use constructs from other domain specific language (for example
relational algebra). Note that users will love a least syntax error
checking in relational algebra sublanguage, so expressions sql(“select
id from ids”) would be less preferable than sql(select id from ids).

ETL directly support these scenarios on syntax side. Complementary "open
compiler" technology is required on implementation side. From what I
see, with xText one would quickly come to phase when answer to customer
request would be “this is too complex to implement in maintainable way”.
Please correct me if I’m wrong here.

On representation, I do not think that it ETL is too limiting. I have
done experiments with imperative language like Java (EJ in samples) and
functional like Haskell (not published because I need to learn Haskell
more to finalize it). The tailored language is usually more readable and
shorter (but sometime ETL based languages are shorter [for example
Pascal]). But result is usually better than LISP’s way and result is the
way better than XML for many cases. Considering advantages of
extensibility, reuse, and automatic error correction, I think that
overhead worth it in long time. Also this is an issue of traction: the
more DSLs are defined in ETL way the acceptable it will look.

I also do not think that simple language import will solve the problem.
Imported language should be usually customized to play well with
environment. For example in state machine example, the reused action
language would be likely extended by statement like “nextstate <state
label>”. Relational algebra DSL embedded into some host language like
Java could require inserting code fragments in host language like
"sql(select id from id where name = #(person.getName().substring(0,14))
and id < 1000)" (to replace prepared statement followed by parameter
binding).

Constantine

Sven Efftinge wrote:
> Hi Constantine,
>
> there are definitely a lot of interesting ideas in your work.
> I think the main difference between our approaches is the way of
> abstraction.
> With xText grammar language one describes a textual language, which is
> mapped on an Ecore model. The abstraction lays in hiding all the little
> tweeking options, and stuff. So the grammars are very simple and readable.
>
> ETL seems to provide concepts for general purpose programming languages
> (with expressions, statements, etc.) and the language descriptions are
> not that readable (IMO and of course the provided examples are more
> complex) Additionally the language seems to be far more complex (indeed
> far more powerful, too)
>
> It's not the goal of xText to define complex languages containing
> expressions and stuff, but to be able to write small structural
> languages (little languages) in about 5 min (this is what my DSLs are in
> about 98% of all cases).
> Additionally it is important (IMO), to define the conrete syntax,
> because the language should be represented the way the user (domain
> expert, who ever that is) wants it to be. I don't want to automatically
> have semicolons and curly brackets everywhere.
>
> I really like the extension mechanism ETL provides (I missed such things
> in common parser generators). I want to add the possibility of grammar
> imports and external rule references to xText so this might relativize
> the missing extensibility you mentioned. Of course it's a different
> concept and not that flexible and more invasive.
>
> The xText editor uses the generated antLR parser, which is not
> incremental. IMO this is not necessary, because the parser is fast enough.
>
> The transformation and constraint component from xText are the languages
> "xtend" respective "check" from openArchitectureWare. they are not EMF
> dependent but can work on EMF models. BTW.: Both languages use the same
> expression sub-language which is another proof for your extensibility
> requirement :-)
>
> regards,
> Sven
>
> Constantine Plotnikov schrieb:
>> Hi,
>>
>> I have the project in the same direction. It is ETL (see
>> http://etl.sourceforge.net/etl-java-0_2_0/doc/readme.html for online
>> documentation and
>> http://sourceforge.net/project/showfiles.php?group_id=153075 for demos
>> and documentation).
>>
>> ETL also allows creating a EMF tree out of text. It is also possible
>> to create other kinds of trees using the framework (like using plain
>> JavaBeans) because ETL provides a generic pull parser. Note that ETL
>> does not requires Java code generation to create parser from grammar
>> definition. The grammar is compiled into state machine in runtime. ETL
>> parser framework component organization is quite similar to one of
>> typical XML parser.
>>
>> However in my project I have focused on extensibility and reusability
>> of textual notation definitions rather than on generality of syntax
>> definition. As result, in ETL is it is possible to define smaller set
>> of surface syntaxes than in xText. Deep in grammar definition
>> pipeline, the grammar is translated to LL(1) form. Additional
>> restrictions come from shared phrase parser and lexer.
>>
>> But on other hand it is possible to combine these languages together
>> and to create new languages from old ones. It is also much simpler to
>> create definitions of operators. See documentation for examples. And I
>> believe for every language definable with LL(k) grammar it is possible
>> to define a language with ETL with the same semantics and which still
>> have a reasonable notation.
>>
>> As I understand the extensibility in xText is more limited as it uses
>> lower-level, but more general constructs to define languages. While
>> wider range of languages could be defined using it, I'm concerned that
>> these languages will become things in themselves. Lack of reuse would
>> not hurt on the small projects. But in larger ones it could be
>> somewhat counterproductive, as do-not-repeat-yourself principle will
>> be violated. Another problem is that lack of reuse (except using
>> "copy/paste" reuse methodology) will lead to languages that differ in
>> minor details like numeric literals, control constructs, etc. even in
>> DSLs created by single organization. It will impossible to select a
>> common sub-language and directly express it as such using grammar
>> definition. This will be barrier for learning and use of DSLs.
>>
>> In addition to tree based parsers ETL support pull parsing to support
>> interpreter read-eval-print loops. I also think that pull parsing will
>> be more convenient for text editors.
>>
>> Note that ETL is in much more early development stages than xText. I
>> am not able to dedicate much time to it as it is still a personal
>> project. There is no generic customizable text editor for the language
>> yet. It is believed that such text editor is possible.
>>
>> I'm also interested if the xText editor supports reparsing of the
>> source code fragments or always parses entire source.
>>
>> There might be some integration directions between ETL and xText:
>> - If editor framework is flexible enough, it can be used to create
>> general customizable ETL text editor.
>> - Transformation and constraint component might be reused if they
>> mostly depend on EMF.
>> - ETL might be used to define some languages used in xText framework.
>> It is possible that syntax for sources provided in presentation could
>> be expressed.
>> - A specialized xText ETL framework could be created that simplifies
>> xText editor creation for ETL-based languages.
>>
>> Constantine
>>
>>
>> Sven Efftinge wrote:
>>> Hi,
>>>
>>> I just met Artem at the JAX conference in Germany, where we talked a
>>> bit about different approaches for textual notations for DSLs.
>>>
>>> I wanted to point you to the xText framework we are currently
>>> developing at openArchitectureWare.
>>> It offers (generates) a textual parser and a textual eclipse editor
>>> based on a grammar described in a simplified BNF-notation.
>>>
>>> The generated parser is based on AntLR and creates a (dynamic) EMF
>>> model. The generated text editor provides syntax highlighting, syntax
>>> checking, constraint checking and an outline view.
>>>
>>> I've written a blog entry showing some more details (and
>>> screenshots):
>>> http://effi-blog.blogspot.com/2006/04/textual-dsl-framework- xtext.html
>>>
>>> The xText framework was also a part of our session at "Eclipse Forum
>>> Europe", which shows the integration of EMF and GMF with oAW (and
>>> xText). Some of you may find that interesting, too.
>>> The slides can be found here:
>>> http://www.eclipse.org/gmt/oaw/doc/s10_gmfAndOAW_slides.pdf
>>>
>>> We are very interested in sharing ideas about textual DSL design and
>>> would be happy if we could contribute some ideas or code to a
>>> possible TMF (Textual Modelling Framework) if there are plans to
>>> create such a project.
>>>
>>>
>>> best regards,
>>> Sven
Re: Textual Modelling Framework? [message #377138 is a reply to message #377134] Thu, 25 May 2006 21:13 Go to previous messageGo to next message
Sven Efftinge is currently offline Sven Efftinge
Messages: 1769
Registered: July 2009
Senior Member
Constantine,

As I said in my previous post, xText is meant to be used for little
structural languages.
If you really want to develop a complex textual language like the one
you describe (state machine with expressions for guards and actions) I
would recommend using a common parser generator (or maybe ETL when it is
more matured).
This is definitely no target language for xText!
Using xText (or GMF) I would describe the guards and actions with Java
against the generated Java code (like we do it in the example).
That makes the development of languages and generators much easier
and the drawback (describing behaviour in another artifact) is not that
big. To be honest, I think it's an advantage, because I can use all the
good tool support from JDT (not just syntax checking).

Developing such complex languages (including good tool support) is no
easy task. Additionally the user has to understand the DSL, so it should
be simple.

Don't get me wrong, I really like many of your ideas. And it is
definitely important that we do more research in this area, but I think
for a Textual Modelling Framework (helping developing DSLs) it's better
to focus on a smaller set of languages so we can describe those in a
more abstract and easy manner.

regards,
Sven



Constantine Plotnikov schrieb:
> Hi Sven,
>
> The ETL grammar definition language also specifies mapping from source
> to AST (that might happen to be EMF/MOF). The simpler grammar definition
> language is possible. I will evaluate it for 0.3 after more languages
> are created and I will start seeing construct usage patterns better. The
> current grammar language lacks any shortcuts and it should definitely
> evolve. Theoretically statement definition language can be of about the
> same complexity as production language in xText.
>
> Some things in the language that increase its verbosity are done to
> support extensibility, but they are possibly too straightforward. For
> example name of statement/operator and name of AST object are different
> to support extensibility (in xText they are the same as I understand).
> This also had allowed defining different mapping to single AST node from
> different contexts. I guess it could have been done in other way using
> “redefine” declaration but that would have put some restrictions on
> extensibility.
>
> Also there is a bit less pressure on verbosity in ETL than in xText. If
> language so big that verbosity is being felt, it is likely that some
> reuse is possible (common expression language, control constructs, etc.).
>
> The state machine is actually a good sample to demonstrate a need for
> extensibility. Initial example is very small and neat. But how long will
> it stay as such when you will start receiving user feedback? Feedback
> will be likely chained like in the following scenario (guesswork from
> experience):
>
> 1. The first thing the users will likely want would be guards on
> transitions. This will lead to expression sublanguage. Possibly an
> existing one like Java or OCL or directly mappable to them.
>
> 2. Then it is very likely that users will want action sublanguage that
> shares some things with expression sublanguage. Specifying a method name
> is a bit too limiting. User will prefer to specify code in place rather
> than in some separate template. And the user will prefer to reference
> external code libraries directly from action language.
>
> 3. Then it is very likely the project with completely different actions
> and expression will want to reuse state machine definition but to use
> own action and expression sublanguage. This will require factoring out
> abstract reusable state machine language (unless copy/paste reuse is
> undertaken).
>
> 4. Then in the action or expression sublanguage they will likely want to
> use constructs from other domain specific language (for example
> relational algebra). Note that users will love a least syntax error
> checking in relational algebra sublanguage, so expressions sql(“select
> id from ids”) would be less preferable than sql(select id from ids).
>
> ETL directly support these scenarios on syntax side. Complementary "open
> compiler" technology is required on implementation side. From what I
> see, with xText one would quickly come to phase when answer to customer
> request would be “this is too complex to implement in maintainable way”.
> Please correct me if I’m wrong here.
>
> On representation, I do not think that it ETL is too limiting. I have
> done experiments with imperative language like Java (EJ in samples) and
> functional like Haskell (not published because I need to learn Haskell
> more to finalize it). The tailored language is usually more readable and
> shorter (but sometime ETL based languages are shorter [for example
> Pascal]). But result is usually better than LISP’s way and result is the
> way better than XML for many cases. Considering advantages of
> extensibility, reuse, and automatic error correction, I think that
> overhead worth it in long time. Also this is an issue of traction: the
> more DSLs are defined in ETL way the acceptable it will look.
>
> I also do not think that simple language import will solve the problem.
> Imported language should be usually customized to play well with
> environment. For example in state machine example, the reused action
> language would be likely extended by statement like “nextstate <state
> label>”. Relational algebra DSL embedded into some host language like
> Java could require inserting code fragments in host language like
> "sql(select id from id where name = #(person.getName().substring(0,14))
> and id < 1000)" (to replace prepared statement followed by parameter
> binding).
>
> Constantine
>
> Sven Efftinge wrote:
>> Hi Constantine,
>>
>> there are definitely a lot of interesting ideas in your work.
>> I think the main difference between our approaches is the way of
>> abstraction.
>> With xText grammar language one describes a textual language, which is
>> mapped on an Ecore model. The abstraction lays in hiding all the
>> little tweeking options, and stuff. So the grammars are very simple
>> and readable.
>>
>> ETL seems to provide concepts for general purpose programming
>> languages (with expressions, statements, etc.) and the language
>> descriptions are not that readable (IMO and of course the provided
>> examples are more complex) Additionally the language seems to be far
>> more complex (indeed far more powerful, too)
>>
>> It's not the goal of xText to define complex languages containing
>> expressions and stuff, but to be able to write small structural
>> languages (little languages) in about 5 min (this is what my DSLs are
>> in about 98% of all cases).
>> Additionally it is important (IMO), to define the conrete syntax,
>> because the language should be represented the way the user (domain
>> expert, who ever that is) wants it to be. I don't want to
>> automatically have semicolons and curly brackets everywhere.
>>
>> I really like the extension mechanism ETL provides (I missed such
>> things in common parser generators). I want to add the possibility of
>> grammar imports and external rule references to xText so this might
>> relativize the missing extensibility you mentioned. Of course it's a
>> different concept and not that flexible and more invasive.
>>
>> The xText editor uses the generated antLR parser, which is not
>> incremental. IMO this is not necessary, because the parser is fast
>> enough.
>>
>> The transformation and constraint component from xText are the
>> languages "xtend" respective "check" from openArchitectureWare. they
>> are not EMF dependent but can work on EMF models. BTW.: Both languages
>> use the same expression sub-language which is another proof for your
>> extensibility requirement :-)
>>
>> regards,
>> Sven
>>
>> Constantine Plotnikov schrieb:
>>> Hi,
>>>
>>> I have the project in the same direction. It is ETL (see
>>> http://etl.sourceforge.net/etl-java-0_2_0/doc/readme.html for online
>>> documentation and
>>> http://sourceforge.net/project/showfiles.php?group_id=153075 for
>>> demos and documentation).
>>>
>>> ETL also allows creating a EMF tree out of text. It is also possible
>>> to create other kinds of trees using the framework (like using plain
>>> JavaBeans) because ETL provides a generic pull parser. Note that ETL
>>> does not requires Java code generation to create parser from grammar
>>> definition. The grammar is compiled into state machine in runtime.
>>> ETL parser framework component organization is quite similar to one
>>> of typical XML parser.
>>>
>>> However in my project I have focused on extensibility and reusability
>>> of textual notation definitions rather than on generality of syntax
>>> definition. As result, in ETL is it is possible to define smaller set
>>> of surface syntaxes than in xText. Deep in grammar definition
>>> pipeline, the grammar is translated to LL(1) form. Additional
>>> restrictions come from shared phrase parser and lexer.
>>>
>>> But on other hand it is possible to combine these languages together
>>> and to create new languages from old ones. It is also much simpler to
>>> create definitions of operators. See documentation for examples. And
>>> I believe for every language definable with LL(k) grammar it is
>>> possible to define a language with ETL with the same semantics and
>>> which still have a reasonable notation.
>>>
>>> As I understand the extensibility in xText is more limited as it uses
>>> lower-level, but more general constructs to define languages. While
>>> wider range of languages could be defined using it, I'm concerned
>>> that these languages will become things in themselves. Lack of reuse
>>> would not hurt on the small projects. But in larger ones it could be
>>> somewhat counterproductive, as do-not-repeat-yourself principle will
>>> be violated. Another problem is that lack of reuse (except using
>>> "copy/paste" reuse methodology) will lead to languages that differ in
>>> minor details like numeric literals, control constructs, etc. even in
>>> DSLs created by single organization. It will impossible to select a
>>> common sub-language and directly express it as such using grammar
>>> definition. This will be barrier for learning and use of DSLs.
>>>
>>> In addition to tree based parsers ETL support pull parsing to support
>>> interpreter read-eval-print loops. I also think that pull parsing
>>> will be more convenient for text editors.
>>>
>>> Note that ETL is in much more early development stages than xText. I
>>> am not able to dedicate much time to it as it is still a personal
>>> project. There is no generic customizable text editor for the
>>> language yet. It is believed that such text editor is possible.
>>>
>>> I'm also interested if the xText editor supports reparsing of the
>>> source code fragments or always parses entire source.
>>>
>>> There might be some integration directions between ETL and xText:
>>> - If editor framework is flexible enough, it can be used to create
>>> general customizable ETL text editor.
>>> - Transformation and constraint component might be reused if they
>>> mostly depend on EMF.
>>> - ETL might be used to define some languages used in xText framework.
>>> It is possible that syntax for sources provided in presentation could
>>> be expressed.
>>> - A specialized xText ETL framework could be created that simplifies
>>> xText editor creation for ETL-based languages.
>>>
>>> Constantine
>>>
>>>
>>> Sven Efftinge wrote:
>>>> Hi,
>>>>
>>>> I just met Artem at the JAX conference in Germany, where we talked a
>>>> bit about different approaches for textual notations for DSLs.
>>>>
>>>> I wanted to point you to the xText framework we are currently
>>>> developing at openArchitectureWare.
>>>> It offers (generates) a textual parser and a textual eclipse editor
>>>> based on a grammar described in a simplified BNF-notation.
>>>>
>>>> The generated parser is based on AntLR and creates a (dynamic) EMF
>>>> model. The generated text editor provides syntax highlighting,
>>>> syntax checking, constraint checking and an outline view.
>>>>
>>>> I've written a blog entry showing some more details (and
>>>> screenshots):
>>>> http://effi-blog.blogspot.com/2006/04/textual-dsl-framework- xtext.html
>>>>
>>>> The xText framework was also a part of our session at "Eclipse Forum
>>>> Europe", which shows the integration of EMF and GMF with oAW (and
>>>> xText). Some of you may find that interesting, too.
>>>> The slides can be found here:
>>>> http://www.eclipse.org/gmt/oaw/doc/s10_gmfAndOAW_slides.pdf
>>>>
>>>> We are very interested in sharing ideas about textual DSL design and
>>>> would be happy if we could contribute some ideas or code to a
>>>> possible TMF (Textual Modelling Framework) if there are plans to
>>>> create such a project.
>>>>
>>>>
>>>> best regards,
>>>> Sven


--
Need professional support on Xtext or Xtend?
Mail to: xtext (at) itemis.com
Twitter : @svenefftinge
Blog : blog.efftinge.de
Re: Textual Modelling Framework? [message #377219 is a reply to message #376408] Sat, 24 June 2006 01:01 Go to previous messageGo to next message
David Williams is currently offline David Williams
Messages: 696
Registered: July 2009
Senior Member
On Tue, 23 May 2006 11:05:18 -0400, Sven Efftinge <sven@efftinge.de> wrote:


>
> We are very interested in sharing ideas about textual DSL design

This sounds like it must be obvious to others on this list, but
what exactly is the advantage of using a "new language" to
describe a DSL instead of, let's say, using XML?
Re: Textual Modelling Framework? [message #377221 is a reply to message #377219] Sat, 24 June 2006 01:55 Go to previous message
Richard Gronback is currently offline Richard Gronback
Messages: 605
Registered: July 2009
Senior Member
Human usability is the primary motivation, I'd say. It's not about creating
a "new language," but providing a concrete syntax for a language that is
more "human friendly" than XML.


On 6/23/06 9:01 PM, in article op.tbmk4agaac05ss@dmw2t23.raleigh.ibm.com,
"David Williams" <david_williams@us.ibm.com> wrote:

> On Tue, 23 May 2006 11:05:18 -0400, Sven Efftinge <sven@efftinge.de> wrote:
>
>
>>
>> We are very interested in sharing ideas about textual DSL design
>
> This sounds like it must be obvious to others on this list, but
> what exactly is the advantage of using a "new language" to
> describe a DSL instead of, let's say, using XML?
>
>
Re: Textual Modelling Framework? [message #565017 is a reply to message #376408] Wed, 24 May 2006 06:55 Go to previous message
Richard Gronback is currently offline Richard Gronback
Messages: 23
Registered: July 2009
Junior Member
Hi Sven,

I recently came across xText while reading this
https://bugs.eclipse.org/bugs/show_bug.cgi?id=141679
Re: Textual Modelling Framework? [message #565038 is a reply to message #376409] Wed, 24 May 2006 09:37 Go to previous message
Richard Gronback is currently offline Richard Gronback
Messages: 605
Registered: July 2009
Senior Member
I guess the remainder of my note didn't make it...

What it said was that you might consider getting together with Artem and
Chris Daly (Emfatic) to work on a project proposal. It's definitely a project
we'd like to see started, as it's on our charter. We look forward to your
contribution :)

Best Regards,
Rich
Re: Textual Modelling Framework? [message #565074 is a reply to message #376410] Wed, 24 May 2006 11:01 Go to previous message
Ed Merks is currently offline Ed Merks
Messages: 26152
Registered: July 2009
Senior Member
Rich,

I was asking Chris for a comparison between this work and his Gymnast
work; there are lots of similarities and a few differences.
Unfortunately, because of his use of ANTLR, Chris' donation is being
held up. I suspect that Sven's work will run into the same legal
barrier. :-( Other projects (OCL) that used ANTLR have switched to
LPG. I'll follow up with Janet to confirm if there is now an absolute
barrier against the use of ANTLR or if perhaps the development work to
migrate to a framework with a more sound pedigree can take place within
the Eclipse open source community rather than being forced outside of it.


Richard Gronback wrote:

> I guess the remainder of my note didn't make it...
>
> What it said was that you might consider getting together with Artem
> and Chris Daly (Emfatic) to work on a project proposal. It's
> definitely a project we'd like to see started, as it's on our
> charter. We look forward to your contribution :)
>
> Best Regards,
> Rich
>
>
related idea at Google Summer of Code [message #565099 is a reply to message #376408] Wed, 24 May 2006 11:22 Go to previous message
Miguel Garcia is currently offline Miguel Garcia
Messages: 40
Registered: July 2009
Member
The following project idea

> Eclipse IDE generator: building on some previous research work (Chris
> Laffra), provide an Eclipse IDE generation environment derived from a
> language grammar. This project would allow the creation of basic support
> of new or existing languages in Eclipse rapidly.

appears listed at
http://wiki.eclipse.org/index.php/Google_Summer_of_Code_2006

Given that applicants were to be notified before May 22nd, 2006 the outcome
should be out already.


Miguel Garcia
Re: Textual Modelling Framework? [message #565130 is a reply to message #376411] Wed, 24 May 2006 11:41 Go to previous message
Srgjan Srepfler is currently offline Srgjan Srepfler
Messages: 33
Registered: July 2009
Member
Ed Merks wrote:
> Rich,
>
> I was asking Chris for a comparison between this work and his Gymnast
> work; there are lots of similarities and a few differences.
> Unfortunately, because of his use of ANTLR, Chris' donation is being
> held up. I suspect that Sven's work will run into the same legal
> barrier. :-( Other projects (OCL) that used ANTLR have switched to
> LPG. I'll follow up with Janet to confirm if there is now an absolute
> barrier against the use of ANTLR or if perhaps the development work to
> migrate to a framework with a more sound pedigree can take place within
> the Eclipse open source community rather than being forced outside of it.
>
>
> Richard Gronback wrote:
>
>> I guess the remainder of my note didn't make it...
>>
>> What it said was that you might consider getting together with Artem
>> and Chris Daly (Emfatic) to work on a project proposal. It's
>> definitely a project we'd like to see started, as it's on our
>> charter. We look forward to your contribution :)
>>
>> Best Regards,
>> Rich
>>
>>
Is ANTLR inferior to LPG or is it a licensing issue?
Srgjan
Re: Textual Modelling Framework? [message #565143 is a reply to message #376413] Wed, 24 May 2006 11:52 Go to previous message
Ed Merks is currently offline Ed Merks
Messages: 26152
Registered: July 2009
Senior Member
Srgjan,

The issue is a legal one in that ANTLR has a poorly tracked pedigree.
I.e., it's not entirely clear where everything in ANTLR came from and
whether all contributors intended their work to be licensed as it is.


Srgjan Srepfler wrote:

> Ed Merks wrote:
>
>> Rich,
>>
>> I was asking Chris for a comparison between this work and his Gymnast
>> work; there are lots of similarities and a few differences.
>> Unfortunately, because of his use of ANTLR, Chris' donation is being
>> held up. I suspect that Sven's work will run into the same legal
>> barrier. :-( Other projects (OCL) that used ANTLR have switched to
>> LPG. I'll follow up with Janet to confirm if there is now an
>> absolute barrier against the use of ANTLR or if perhaps the
>> development work to migrate to a framework with a more sound pedigree
>> can take place within the Eclipse open source community rather than
>> being forced outside of it.
>>
>>
>> Richard Gronback wrote:
>>
>>> I guess the remainder of my note didn't make it...
>>>
>>> What it said was that you might consider getting together with Artem
>>> and Chris Daly (Emfatic) to work on a project proposal. It's
>>> definitely a project we'd like to see started, as it's on our
>>> charter. We look forward to your contribution :)
>>>
>>> Best Regards,
>>> Rich
>>>
>>>
> Is ANTLR inferior to LPG or is it a licensing issue?
> Srgjan
Re: Textual Modelling Framework? [message #565170 is a reply to message #376408] Wed, 24 May 2006 12:47 Go to previous message
Constantine Plotnikov is currently offline Constantine Plotnikov
Messages: 45
Registered: July 2009
Member
Hi,

I have the project in the same direction. It is ETL (see
http://etl.sourceforge.net/etl-java-0_2_0/doc/readme.html for online
documentation and
http://sourceforge.net/project/showfiles.php?group_id=153075 for demos
and documentation).

ETL also allows creating a EMF tree out of text. It is also possible to
create other kinds of trees using the framework (like using plain
JavaBeans) because ETL provides a generic pull parser. Note that ETL
does not requires Java code generation to create parser from grammar
definition. The grammar is compiled into state machine in runtime. ETL
parser framework component organization is quite similar to one of
typical XML parser.

However in my project I have focused on extensibility and reusability of
textual notation definitions rather than on generality of syntax
definition. As result, in ETL is it is possible to define smaller set of
surface syntaxes than in xText. Deep in grammar definition pipeline, the
grammar is translated to LL(1) form. Additional restrictions come from
shared phrase parser and lexer.

But on other hand it is possible to combine these languages together and
to create new languages from old ones. It is also much simpler to create
definitions of operators. See documentation for examples. And I believe
for every language definable with LL(k) grammar it is possible to define
a language with ETL with the same semantics and which still have a
reasonable notation.

As I understand the extensibility in xText is more limited as it uses
lower-level, but more general constructs to define languages. While
wider range of languages could be defined using it, I'm concerned that
these languages will become things in themselves. Lack of reuse would
not hurt on the small projects. But in larger ones it could be somewhat
counterproductive, as do-not-repeat-yourself principle will be violated.
Another problem is that lack of reuse (except using "copy/paste" reuse
methodology) will lead to languages that differ in minor details like
numeric literals, control constructs, etc. even in DSLs created by
single organization. It will impossible to select a common sub-language
and directly express it as such using grammar definition. This will be
barrier for learning and use of DSLs.

In addition to tree based parsers ETL support pull parsing to support
interpreter read-eval-print loops. I also think that pull parsing will
be more convenient for text editors.

Note that ETL is in much more early development stages than xText. I am
not able to dedicate much time to it as it is still a personal project.
There is no generic customizable text editor for the language yet. It is
believed that such text editor is possible.

I'm also interested if the xText editor supports reparsing of the source
code fragments or always parses entire source.

There might be some integration directions between ETL and xText:
- If editor framework is flexible enough, it can be used to create
general customizable ETL text editor.
- Transformation and constraint component might be reused if they mostly
depend on EMF.
- ETL might be used to define some languages used in xText framework. It
is possible that syntax for sources provided in presentation could be
expressed.
- A specialized xText ETL framework could be created that simplifies
xText editor creation for ETL-based languages.

Constantine


Sven Efftinge wrote:
> Hi,
>
> I just met Artem at the JAX conference in Germany, where we talked a bit
> about different approaches for textual notations for DSLs.
>
> I wanted to point you to the xText framework we are currently developing
> at openArchitectureWare.
> It offers (generates) a textual parser and a textual eclipse editor
> based on a grammar described in a simplified BNF-notation.
>
> The generated parser is based on AntLR and creates a (dynamic) EMF
> model. The generated text editor provides syntax highlighting, syntax
> checking, constraint checking and an outline view.
>
> I've written a blog entry showing some more details (and screenshots):
> http://effi-blog.blogspot.com/2006/04/textual-dsl-framework- xtext.html
>
> The xText framework was also a part of our session at "Eclipse Forum
> Europe", which shows the integration of EMF and GMF with oAW (and
> xText). Some of you may find that interesting, too.
> The slides can be found here:
> http://www.eclipse.org/gmt/oaw/doc/s10_gmfAndOAW_slides.pdf
>
> We are very interested in sharing ideas about textual DSL design and
> would be happy if we could contribute some ideas or code to a possible
> TMF (Textual Modelling Framework) if there are plans to create such a
> project.
>
>
> best regards,
> Sven
Re: Textual Modelling Framework? [message #565199 is a reply to message #376415] Wed, 24 May 2006 15:15 Go to previous message
Sven Efftinge is currently offline Sven Efftinge
Messages: 1769
Registered: July 2009
Senior Member
Hi Constantine,

there are definitely a lot of interesting ideas in your work.
I think the main difference between our approaches is the way of
abstraction.
With xText grammar language one describes a textual language, which is
mapped on an Ecore model. The abstraction lays in hiding all the little
tweeking options, and stuff. So the grammars are very simple and readable.

ETL seems to provide concepts for general purpose programming languages
(with expressions, statements, etc.) and the language descriptions are
not that readable (IMO and of course the provided examples are more
complex) Additionally the language seems to be far more complex (indeed
far more powerful, too)

It's not the goal of xText to define complex languages containing
expressions and stuff, but to be able to write small structural
languages (little languages) in about 5 min (this is what my DSLs are in
about 98% of all cases).
Additionally it is important (IMO), to define the conrete syntax,
because the language should be represented the way the user (domain
expert, who ever that is) wants it to be. I don't want to automatically
have semicolons and curly brackets everywhere.

I really like the extension mechanism ETL provides (I missed such things
in common parser generators). I want to add the possibility of grammar
imports and external rule references to xText so this might relativize
the missing extensibility you mentioned. Of course it's a different
concept and not that flexible and more invasive.

The xText editor uses the generated antLR parser, which is not
incremental. IMO this is not necessary, because the parser is fast enough.

The transformation and constraint component from xText are the languages
"xtend" respective "check" from openArchitectureWare. they are not EMF
dependent but can work on EMF models. BTW.: Both languages use the same
expression sub-language which is another proof for your extensibility
requirement :-)

regards,
Sven

Constantine Plotnikov schrieb:
> Hi,
>
> I have the project in the same direction. It is ETL (see
> http://etl.sourceforge.net/etl-java-0_2_0/doc/readme.html for online
> documentation and
> http://sourceforge.net/project/showfiles.php?group_id=153075 for demos
> and documentation).
>
> ETL also allows creating a EMF tree out of text. It is also possible to
> create other kinds of trees using the framework (like using plain
> JavaBeans) because ETL provides a generic pull parser. Note that ETL
> does not requires Java code generation to create parser from grammar
> definition. The grammar is compiled into state machine in runtime. ETL
> parser framework component organization is quite similar to one of
> typical XML parser.
>
> However in my project I have focused on extensibility and reusability of
> textual notation definitions rather than on generality of syntax
> definition. As result, in ETL is it is possible to define smaller set of
> surface syntaxes than in xText. Deep in grammar definition pipeline, the
> grammar is translated to LL(1) form. Additional restrictions come from
> shared phrase parser and lexer.
>
> But on other hand it is possible to combine these languages together and
> to create new languages from old ones. It is also much simpler to create
> definitions of operators. See documentation for examples. And I believe
> for every language definable with LL(k) grammar it is possible to define
> a language with ETL with the same semantics and which still have a
> reasonable notation.
>
> As I understand the extensibility in xText is more limited as it uses
> lower-level, but more general constructs to define languages. While
> wider range of languages could be defined using it, I'm concerned that
> these languages will become things in themselves. Lack of reuse would
> not hurt on the small projects. But in larger ones it could be somewhat
> counterproductive, as do-not-repeat-yourself principle will be violated.
> Another problem is that lack of reuse (except using "copy/paste" reuse
> methodology) will lead to languages that differ in minor details like
> numeric literals, control constructs, etc. even in DSLs created by
> single organization. It will impossible to select a common sub-language
> and directly express it as such using grammar definition. This will be
> barrier for learning and use of DSLs.
>
> In addition to tree based parsers ETL support pull parsing to support
> interpreter read-eval-print loops. I also think that pull parsing will
> be more convenient for text editors.
>
> Note that ETL is in much more early development stages than xText. I am
> not able to dedicate much time to it as it is still a personal project.
> There is no generic customizable text editor for the language yet. It is
> believed that such text editor is possible.
>
> I'm also interested if the xText editor supports reparsing of the source
> code fragments or always parses entire source.
>
> There might be some integration directions between ETL and xText:
> - If editor framework is flexible enough, it can be used to create
> general customizable ETL text editor.
> - Transformation and constraint component might be reused if they mostly
> depend on EMF.
> - ETL might be used to define some languages used in xText framework. It
> is possible that syntax for sources provided in presentation could be
> expressed.
> - A specialized xText ETL framework could be created that simplifies
> xText editor creation for ETL-based languages.
>
> Constantine
>
>
> Sven Efftinge wrote:
>> Hi,
>>
>> I just met Artem at the JAX conference in Germany, where we talked a
>> bit about different approaches for textual notations for DSLs.
>>
>> I wanted to point you to the xText framework we are currently
>> developing at openArchitectureWare.
>> It offers (generates) a textual parser and a textual eclipse editor
>> based on a grammar described in a simplified BNF-notation.
>>
>> The generated parser is based on AntLR and creates a (dynamic) EMF
>> model. The generated text editor provides syntax highlighting, syntax
>> checking, constraint checking and an outline view.
>>
>> I've written a blog entry showing some more details (and screenshots):
>> http://effi-blog.blogspot.com/2006/04/textual-dsl-framework- xtext.html
>>
>> The xText framework was also a part of our session at "Eclipse Forum
>> Europe", which shows the integration of EMF and GMF with oAW (and
>> xText). Some of you may find that interesting, too.
>> The slides can be found here:
>> http://www.eclipse.org/gmt/oaw/doc/s10_gmfAndOAW_slides.pdf
>>
>> We are very interested in sharing ideas about textual DSL design and
>> would be happy if we could contribute some ideas or code to a possible
>> TMF (Textual Modelling Framework) if there are plans to create such a
>> project.
>>
>>
>> best regards,
>> Sven


--
Need professional support on Xtext or Xtend?
Mail to: xtext (at) itemis.com
Twitter : @svenefftinge
Blog : blog.efftinge.de
Re: Textual Modelling Framework? [message #565230 is a reply to message #376411] Wed, 24 May 2006 15:26 Go to previous message
Sven Efftinge is currently offline Sven Efftinge
Messages: 1769
Registered: July 2009
Senior Member
I also had a look at some presentation slides about Gymnast and indeed
the approaches are very similar.
regarding the license issue:
I thought about switching to Antlr 3 (when it is in beta).
It will be released under BSD.
I don't know LPG but I will have a look at it...

Sven

Ed Merks schrieb:
> Rich,
>
> I was asking Chris for a comparison between this work and his Gymnast
> work; there are lots of similarities and a few differences.
> Unfortunately, because of his use of ANTLR, Chris' donation is being
> held up. I suspect that Sven's work will run into the same legal
> barrier. :-( Other projects (OCL) that used ANTLR have switched to
> LPG. I'll follow up with Janet to confirm if there is now an absolute
> barrier against the use of ANTLR or if perhaps the development work to
> migrate to a framework with a more sound pedigree can take place within
> the Eclipse open source community rather than being forced outside of it.
>
>
> Richard Gronback wrote:
>
>> I guess the remainder of my note didn't make it...
>>
>> What it said was that you might consider getting together with Artem
>> and Chris Daly (Emfatic) to work on a project proposal. It's
>> definitely a project we'd like to see started, as it's on our
>> charter. We look forward to your contribution :)
>>
>> Best Regards,
>> Rich
>>
>>


--
Need professional support on Xtext or Xtend?
Mail to: xtext (at) itemis.com
Twitter : @svenefftinge
Blog : blog.efftinge.de
Re: Textual Modelling Framework? [message #565268 is a reply to message #376417] Wed, 24 May 2006 18:15 Go to previous message
Ed Merks is currently offline Ed Merks
Messages: 26152
Registered: July 2009
Senior Member
This is a multi-part message in MIME format.
--------------010905010501070005010905
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Sven,

This is what Janet Campbell had to say:

To date I have not approved ANTLR for use in any Eclipse project.
Contributions are reviewed and approved/declined based on the
results of the investigation of the code. In the case of ANTLR
pedigree concerns were identified and use of ANTLR in Eclipse was
not approved. A revised version of ANTLR with a cleaner IP trail is
due for release this summer at which time we can revisit the issue.
Please see: http://www.antlr.org/v3/index.html

LPG has definitely been approved; whether the next version of ANTLR will
be approved will depend on the quality of the legal audit trail for all
its contributions...


Sven Efftinge wrote:

> I also had a look at some presentation slides about Gymnast and indeed
> the approaches are very similar.
> regarding the license issue:
> I thought about switching to Antlr 3 (when it is in beta).
> It will be released under BSD.
> I don't know LPG but I will have a look at it...
>
> Sven
>
> Ed Merks schrieb:
>
>> Rich,
>>
>> I was asking Chris for a comparison between this work and his Gymnast
>> work; there are lots of similarities and a few differences.
>> Unfortunately, because of his use of ANTLR, Chris' donation is being
>> held up. I suspect that Sven's work will run into the same legal
>> barrier. :-( Other projects (OCL) that used ANTLR have switched to
>> LPG. I'll follow up with Janet to confirm if there is now an
>> absolute barrier against the use of ANTLR or if perhaps the
>> development work to migrate to a framework with a more sound pedigree
>> can take place within the Eclipse open source community rather than
>> being forced outside of it.
>>
>>
>> Richard Gronback wrote:
>>
>>> I guess the remainder of my note didn't make it...
>>>
>>> What it said was that you might consider getting together with Artem
>>> and Chris Daly (Emfatic) to work on a project proposal. It's
>>> definitely a project we'd like to see started, as it's on our
>>> charter. We look forward to your contribution :)
>>>
>>> Best Regards,
>>> Rich
>>>
>>>


--------------010905010501070005010905
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Sven,<br>
<br>
This is what Janet Campbell had to say: <br>
<blockquote>To date I have not approved ANTLR for use in any Eclipse
project.&nbsp; Contributions are reviewed and approved/declined based on the
results of the investigation of the code.&nbsp; In the case of ANTLR
pedigree concerns were identified and use of ANTLR in Eclipse was not
approved.&nbsp; A revised version of ANTLR with a cleaner IP trail is due
for release this summer at which time we can revisit the issue.&nbsp; Please
see:&nbsp; <a class="moz-txt-link-freetext" href="http://www.antlr.org/v3/index.html">http://www.antlr.org/v3/index.html</a>. <br>
</blockquote>
LPG has definitely been approved; whether the next version of ANTLR
will be approved will depend on the quality of the legal audit trail
for all its contributions...<br>
<br>
<br>
Sven Efftinge wrote:
<blockquote cite="mide51tul$a1g$1@utils.eclipse.org" type="cite">I also
had a look at some presentation slides about Gymnast and indeed the
approaches are very similar.
<br>
regarding the license issue:
<br>
I thought about switching to Antlr 3 (when it is in beta).
<br>
It will be released under BSD.
<br>
I don't know LPG but I will have a look at it...
<br>
<br>
Sven
<br>
<br>
Ed Merks schrieb:
<br>
<blockquote type="cite">Rich,
<br>
<br>
I was asking Chris for a comparison between this work and his Gymnast
work; there are lots of similarities and a few differences.&nbsp;
Unfortunately, because of his use of ANTLR, Chris' donation is being
held up.&nbsp; I suspect that Sven's work will run into the same legal
barrier.&nbsp; :-(&nbsp; Other projects (OCL) that used ANTLR have switched to
LPG.&nbsp; I'll follow up with Janet to confirm if there is now an absolute
barrier against the use of ANTLR or if perhaps the development work to
migrate to a framework with a more sound pedigree can take place within
the Eclipse open source community rather than being forced outside of
it.
<br>
<br>
<br>
Richard Gronback wrote:
<br>
<br>
<blockquote type="cite">I guess the remainder of my note didn't
make it...
<br>
<br>
What it said was that you might consider getting together with Artem
and Chris Daly (Emfatic) to work on a project proposal.&nbsp; It's
definitely a project we'd like to see started, as it's on our charter.&nbsp;
We look forward to your contribution :)
<br>
<br>
Best Regards,
<br>
Rich
<br>
<br>
<br>
</blockquote>
</blockquote>
</blockquote>
<br>
</body>
</html>

--------------010905010501070005010905--
Re: Textual Modelling Framework? [message #572816 is a reply to message #376416] Thu, 25 May 2006 15:16 Go to previous message
Constantine Plotnikov is currently offline Constantine Plotnikov
Messages: 45
Registered: July 2009
Member
Hi Sven,

The ETL grammar definition language also specifies mapping from source
to AST (that might happen to be EMF/MOF). The simpler grammar definition
language is possible. I will evaluate it for 0.3 after more languages
are created and I will start seeing construct usage patterns better. The
current grammar language lacks any shortcuts and it should definitely
evolve. Theoretically statement definition language can be of about the
same complexity as production language in xText.

Some things in the language that increase its verbosity are done to
support extensibility, but they are possibly too straightforward. For
example name of statement/operator and name of AST object are different
to support extensibility (in xText they are the same as I understand).
This also had allowed defining different mapping to single AST node from
different contexts. I guess it could have been done in other way using
“redefine” declaration but that would have put some restrictions on
extensibility.

Also there is a bit less pressure on verbosity in ETL than in xText. If
language so big that verbosity is being felt, it is likely that some
reuse is possible (common expression language, control constructs, etc.).

The state machine is actually a good sample to demonstrate a need for
extensibility. Initial example is very small and neat. But how long will
it stay as such when you will start receiving user feedback? Feedback
will be likely chained like in the following scenario (guesswork from
experience):

1. The first thing the users will likely want would be guards on
transitions. This will lead to expression sublanguage. Possibly an
existing one like Java or OCL or directly mappable to them.

2. Then it is very likely that users will want action sublanguage that
shares some things with expression sublanguage. Specifying a method name
is a bit too limiting. User will prefer to specify code in place rather
than in some separate template. And the user will prefer to reference
external code libraries directly from action language.

3. Then it is very likely the project with completely different actions
and expression will want to reuse state machine definition but to use
own action and expression sublanguage. This will require factoring out
abstract reusable state machine language (unless copy/paste reuse is
undertaken).

4. Then in the action or expression sublanguage they will likely want to
use constructs from other domain specific language (for example
relational algebra). Note that users will love a least syntax error
checking in relational algebra sublanguage, so expressions sql(“select
id from ids”) would be less preferable than sql(select id from ids).

ETL directly support these scenarios on syntax side. Complementary "open
compiler" technology is required on implementation side. From what I
see, with xText one would quickly come to phase when answer to customer
request would be “this is too complex to implement in maintainable way”.
Please correct me if I’m wrong here.

On representation, I do not think that it ETL is too limiting. I have
done experiments with imperative language like Java (EJ in samples) and
functional like Haskell (not published because I need to learn Haskell
more to finalize it). The tailored language is usually more readable and
shorter (but sometime ETL based languages are shorter [for example
Pascal]). But result is usually better than LISP’s way and result is the
way better than XML for many cases. Considering advantages of
extensibility, reuse, and automatic error correction, I think that
overhead worth it in long time. Also this is an issue of traction: the
more DSLs are defined in ETL way the acceptable it will look.

I also do not think that simple language import will solve the problem.
Imported language should be usually customized to play well with
environment. For example in state machine example, the reused action
language would be likely extended by statement like “nextstate <state
label>”. Relational algebra DSL embedded into some host language like
Java could require inserting code fragments in host language like
"sql(select id from id where name = #(person.getName().substring(0,14))
and id < 1000)" (to replace prepared statement followed by parameter
binding).

Constantine

Sven Efftinge wrote:
> Hi Constantine,
>
> there are definitely a lot of interesting ideas in your work.
> I think the main difference between our approaches is the way of
> abstraction.
> With xText grammar language one describes a textual language, which is
> mapped on an Ecore model. The abstraction lays in hiding all the little
> tweeking options, and stuff. So the grammars are very simple and readable.
>
> ETL seems to provide concepts for general purpose programming languages
> (with expressions, statements, etc.) and the language descriptions are
> not that readable (IMO and of course the provided examples are more
> complex) Additionally the language seems to be far more complex (indeed
> far more powerful, too)
>
> It's not the goal of xText to define complex languages containing
> expressions and stuff, but to be able to write small structural
> languages (little languages) in about 5 min (this is what my DSLs are in
> about 98% of all cases).
> Additionally it is important (IMO), to define the conrete syntax,
> because the language should be represented the way the user (domain
> expert, who ever that is) wants it to be. I don't want to automatically
> have semicolons and curly brackets everywhere.
>
> I really like the extension mechanism ETL provides (I missed such things
> in common parser generators). I want to add the possibility of grammar
> imports and external rule references to xText so this might relativize
> the missing extensibility you mentioned. Of course it's a different
> concept and not that flexible and more invasive.
>
> The xText editor uses the generated antLR parser, which is not
> incremental. IMO this is not necessary, because the parser is fast enough.
>
> The transformation and constraint component from xText are the languages
> "xtend" respective "check" from openArchitectureWare. they are not EMF
> dependent but can work on EMF models. BTW.: Both languages use the same
> expression sub-language which is another proof for your extensibility
> requirement :-)
>
> regards,
> Sven
>
> Constantine Plotnikov schrieb:
>> Hi,
>>
>> I have the project in the same direction. It is ETL (see
>> http://etl.sourceforge.net/etl-java-0_2_0/doc/readme.html for online
>> documentation and
>> http://sourceforge.net/project/showfiles.php?group_id=153075 for demos
>> and documentation).
>>
>> ETL also allows creating a EMF tree out of text. It is also possible
>> to create other kinds of trees using the framework (like using plain
>> JavaBeans) because ETL provides a generic pull parser. Note that ETL
>> does not requires Java code generation to create parser from grammar
>> definition. The grammar is compiled into state machine in runtime. ETL
>> parser framework component organization is quite similar to one of
>> typical XML parser.
>>
>> However in my project I have focused on extensibility and reusability
>> of textual notation definitions rather than on generality of syntax
>> definition. As result, in ETL is it is possible to define smaller set
>> of surface syntaxes than in xText. Deep in grammar definition
>> pipeline, the grammar is translated to LL(1) form. Additional
>> restrictions come from shared phrase parser and lexer.
>>
>> But on other hand it is possible to combine these languages together
>> and to create new languages from old ones. It is also much simpler to
>> create definitions of operators. See documentation for examples. And I
>> believe for every language definable with LL(k) grammar it is possible
>> to define a language with ETL with the same semantics and which still
>> have a reasonable notation.
>>
>> As I understand the extensibility in xText is more limited as it uses
>> lower-level, but more general constructs to define languages. While
>> wider range of languages could be defined using it, I'm concerned that
>> these languages will become things in themselves. Lack of reuse would
>> not hurt on the small projects. But in larger ones it could be
>> somewhat counterproductive, as do-not-repeat-yourself principle will
>> be violated. Another problem is that lack of reuse (except using
>> "copy/paste" reuse methodology) will lead to languages that differ in
>> minor details like numeric literals, control constructs, etc. even in
>> DSLs created by single organization. It will impossible to select a
>> common sub-language and directly express it as such using grammar
>> definition. This will be barrier for learning and use of DSLs.
>>
>> In addition to tree based parsers ETL support pull parsing to support
>> interpreter read-eval-print loops. I also think that pull parsing will
>> be more convenient for text editors.
>>
>> Note that ETL is in much more early development stages than xText. I
>> am not able to dedicate much time to it as it is still a personal
>> project. There is no generic customizable text editor for the language
>> yet. It is believed that such text editor is possible.
>>
>> I'm also interested if the xText editor supports reparsing of the
>> source code fragments or always parses entire source.
>>
>> There might be some integration directions between ETL and xText:
>> - If editor framework is flexible enough, it can be used to create
>> general customizable ETL text editor.
>> - Transformation and constraint component might be reused if they
>> mostly depend on EMF.
>> - ETL might be used to define some languages used in xText framework.
>> It is possible that syntax for sources provided in presentation could
>> be expressed.
>> - A specialized xText ETL framework could be created that simplifies
>> xText editor creation for ETL-based languages.
>>
>> Constantine
>>
>>
>> Sven Efftinge wrote:
>>> Hi,
>>>
>>> I just met Artem at the JAX conference in Germany, where we talked a
>>> bit about different approaches for textual notations for DSLs.
>>>
>>> I wanted to point you to the xText framework we are currently
>>> developing at openArchitectureWare.
>>> It offers (generates) a textual parser and a textual eclipse editor
>>> based on a grammar described in a simplified BNF-notation.
>>>
>>> The generated parser is based on AntLR and creates a (dynamic) EMF
>>> model. The generated text editor provides syntax highlighting, syntax
>>> checking, constraint checking and an outline view.
>>>
>>> I've written a blog entry showing some more details (and
>>> screenshots):
>>> http://effi-blog.blogspot.com/2006/04/textual-dsl-framework- xtext.html
>>>
>>> The xText framework was also a part of our session at "Eclipse Forum
>>> Europe", which shows the integration of EMF and GMF with oAW (and
>>> xText). Some of you may find that interesting, too.
>>> The slides can be found here:
>>> http://www.eclipse.org/gmt/oaw/doc/s10_gmfAndOAW_slides.pdf
>>>
>>> We are very interested in sharing ideas about textual DSL design and
>>> would be happy if we could contribute some ideas or code to a
>>> possible TMF (Textual Modelling Framework) if there are plans to
>>> create such a project.
>>>
>>>
>>> best regards,
>>> Sven
Re: Textual Modelling Framework? [message #572859 is a reply to message #377134] Thu, 25 May 2006 21:13 Go to previous message
Sven Efftinge is currently offline Sven Efftinge
Messages: 1769
Registered: July 2009
Senior Member
Constantine,

As I said in my previous post, xText is meant to be used for little
structural languages.
If you really want to develop a complex textual language like the one
you describe (state machine with expressions for guards and actions) I
would recommend using a common parser generator (or maybe ETL when it is
more matured).
This is definitely no target language for xText!
Using xText (or GMF) I would describe the guards and actions with Java
against the generated Java code (like we do it in the example).
That makes the development of languages and generators much easier
and the drawback (describing behaviour in another artifact) is not that
big. To be honest, I think it's an advantage, because I can use all the
good tool support from JDT (not just syntax checking).

Developing such complex languages (including good tool support) is no
easy task. Additionally the user has to understand the DSL, so it should
be simple.

Don't get me wrong, I really like many of your ideas. And it is
definitely important that we do more research in this area, but I think
for a Textual Modelling Framework (helping developing DSLs) it's better
to focus on a smaller set of languages so we can describe those in a
more abstract and easy manner.

regards,
Sven



Constantine Plotnikov schrieb:
> Hi Sven,
>
> The ETL grammar definition language also specifies mapping from source
> to AST (that might happen to be EMF/MOF). The simpler grammar definition
> language is possible. I will evaluate it for 0.3 after more languages
> are created and I will start seeing construct usage patterns better. The
> current grammar language lacks any shortcuts and it should definitely
> evolve. Theoretically statement definition language can be of about the
> same complexity as production language in xText.
>
> Some things in the language that increase its verbosity are done to
> support extensibility, but they are possibly too straightforward. For
> example name of statement/operator and name of AST object are different
> to support extensibility (in xText they are the same as I understand).
> This also had allowed defining different mapping to single AST node from
> different contexts. I guess it could have been done in other way using
> “redefine” declaration but that would have put some restrictions on
> extensibility.
>
> Also there is a bit less pressure on verbosity in ETL than in xText. If
> language so big that verbosity is being felt, it is likely that some
> reuse is possible (common expression language, control constructs, etc.).
>
> The state machine is actually a good sample to demonstrate a need for
> extensibility. Initial example is very small and neat. But how long will
> it stay as such when you will start receiving user feedback? Feedback
> will be likely chained like in the following scenario (guesswork from
> experience):
>
> 1. The first thing the users will likely want would be guards on
> transitions. This will lead to expression sublanguage. Possibly an
> existing one like Java or OCL or directly mappable to them.
>
> 2. Then it is very likely that users will want action sublanguage that
> shares some things with expression sublanguage. Specifying a method name
> is a bit too limiting. User will prefer to specify code in place rather
> than in some separate template. And the user will prefer to reference
> external code libraries directly from action language.
>
> 3. Then it is very likely the project with completely different actions
> and expression will want to reuse state machine definition but to use
> own action and expression sublanguage. This will require factoring out
> abstract reusable state machine language (unless copy/paste reuse is
> undertaken).
>
> 4. Then in the action or expression sublanguage they will likely want to
> use constructs from other domain specific language (for example
> relational algebra). Note that users will love a least syntax error
> checking in relational algebra sublanguage, so expressions sql(“select
> id from ids”) would be less preferable than sql(select id from ids).
>
> ETL directly support these scenarios on syntax side. Complementary "open
> compiler" technology is required on implementation side. From what I
> see, with xText one would quickly come to phase when answer to customer
> request would be “this is too complex to implement in maintainable way”.
> Please correct me if I’m wrong here.
>
> On representation, I do not think that it ETL is too limiting. I have
> done experiments with imperative language like Java (EJ in samples) and
> functional like Haskell (not published because I need to learn Haskell
> more to finalize it). The tailored language is usually more readable and
> shorter (but sometime ETL based languages are shorter [for example
> Pascal]). But result is usually better than LISP’s way and result is the
> way better than XML for many cases. Considering advantages of
> extensibility, reuse, and automatic error correction, I think that
> overhead worth it in long time. Also this is an issue of traction: the
> more DSLs are defined in ETL way the acceptable it will look.
>
> I also do not think that simple language import will solve the problem.
> Imported language should be usually customized to play well with
> environment. For example in state machine example, the reused action
> language would be likely extended by statement like “nextstate <state
> label>”. Relational algebra DSL embedded into some host language like
> Java could require inserting code fragments in host language like
> "sql(select id from id where name = #(person.getName().substring(0,14))
> and id < 1000)" (to replace prepared statement followed by parameter
> binding).
>
> Constantine
>
> Sven Efftinge wrote:
>> Hi Constantine,
>>
>> there are definitely a lot of interesting ideas in your work.
>> I think the main difference between our approaches is the way of
>> abstraction.
>> With xText grammar language one describes a textual language, which is
>> mapped on an Ecore model. The abstraction lays in hiding all the
>> little tweeking options, and stuff. So the grammars are very simple
>> and readable.
>>
>> ETL seems to provide concepts for general purpose programming
>> languages (with expressions, statements, etc.) and the language
>> descriptions are not that readable (IMO and of course the provided
>> examples are more complex) Additionally the language seems to be far
>> more complex (indeed far more powerful, too)
>>
>> It's not the goal of xText to define complex languages containing
>> expressions and stuff, but to be able to write small structural
>> languages (little languages) in about 5 min (this is what my DSLs are
>> in about 98% of all cases).
>> Additionally it is important (IMO), to define the conrete syntax,
>> because the language should be represented the way the user (domain
>> expert, who ever that is) wants it to be. I don't want to
>> automatically have semicolons and curly brackets everywhere.
>>
>> I really like the extension mechanism ETL provides (I missed such
>> things in common parser generators). I want to add the possibility of
>> grammar imports and external rule references to xText so this might
>> relativize the missing extensibility you mentioned. Of course it's a
>> different concept and not that flexible and more invasive.
>>
>> The xText editor uses the generated antLR parser, which is not
>> incremental. IMO this is not necessary, because the parser is fast
>> enough.
>>
>> The transformation and constraint component from xText are the
>> languages "xtend" respective "check" from openArchitectureWare. they
>> are not EMF dependent but can work on EMF models. BTW.: Both languages
>> use the same expression sub-language which is another proof for your
>> extensibility requirement :-)
>>
>> regards,
>> Sven
>>
>> Constantine Plotnikov schrieb:
>>> Hi,
>>>
>>> I have the project in the same direction. It is ETL (see
>>> http://etl.sourceforge.net/etl-java-0_2_0/doc/readme.html for online
>>> documentation and
>>> http://sourceforge.net/project/showfiles.php?group_id=153075 for
>>> demos and documentation).
>>>
>>> ETL also allows creating a EMF tree out of text. It is also possible
>>> to create other kinds of trees using the framework (like using plain
>>> JavaBeans) because ETL provides a generic pull parser. Note that ETL
>>> does not requires Java code generation to create parser from grammar
>>> definition. The grammar is compiled into state machine in runtime.
>>> ETL parser framework component organization is quite similar to one
>>> of typical XML parser.
>>>
>>> However in my project I have focused on extensibility and reusability
>>> of textual notation definitions rather than on generality of syntax
>>> definition. As result, in ETL is it is possible to define smaller set
>>> of surface syntaxes than in xText. Deep in grammar definition
>>> pipeline, the grammar is translated to LL(1) form. Additional
>>> restrictions come from shared phrase parser and lexer.
>>>
>>> But on other hand it is possible to combine these languages together
>>> and to create new languages from old ones. It is also much simpler to
>>> create definitions of operators. See documentation for examples. And
>>> I believe for every language definable with LL(k) grammar it is
>>> possible to define a language with ETL with the same semantics and
>>> which still have a reasonable notation.
>>>
>>> As I understand the extensibility in xText is more limited as it uses
>>> lower-level, but more general constructs to define languages. While
>>> wider range of languages could be defined using it, I'm concerned
>>> that these languages will become things in themselves. Lack of reuse
>>> would not hurt on the small projects. But in larger ones it could be
>>> somewhat counterproductive, as do-not-repeat-yourself principle will
>>> be violated. Another problem is that lack of reuse (except using
>>> "copy/paste" reuse methodology) will lead to languages that differ in
>>> minor details like numeric literals, control constructs, etc. even in
>>> DSLs created by single organization. It will impossible to select a
>>> common sub-language and directly express it as such using grammar
>>> definition. This will be barrier for learning and use of DSLs.
>>>
>>> In addition to tree based parsers ETL support pull parsing to support
>>> interpreter read-eval-print loops. I also think that pull parsing
>>> will be more convenient for text editors.
>>>
>>> Note that ETL is in much more early development stages than xText. I
>>> am not able to dedicate much time to it as it is still a personal
>>> project. There is no generic customizable text editor for the
>>> language yet. It is believed that such text editor is possible.
>>>
>>> I'm also interested if the xText editor supports reparsing of the
>>> source code fragments or always parses entire source.
>>>
>>> There might be some integration directions between ETL and xText:
>>> - If editor framework is flexible enough, it can be used to create
>>> general customizable ETL text editor.
>>> - Transformation and constraint component might be reused if they
>>> mostly depend on EMF.
>>> - ETL might be used to define some languages used in xText framework.
>>> It is possible that syntax for sources provided in presentation could
>>> be expressed.
>>> - A specialized xText ETL framework could be created that simplifies
>>> xText editor creation for ETL-based languages.
>>>
>>> Constantine
>>>
>>>
>>> Sven Efftinge wrote:
>>>> Hi,
>>>>
>>>> I just met Artem at the JAX conference in Germany, where we talked a
>>>> bit about different approaches for textual notations for DSLs.
>>>>
>>>> I wanted to point you to the xText framework we are currently
>>>> developing at openArchitectureWare.
>>>> It offers (generates) a textual parser and a textual eclipse editor
>>>> based on a grammar described in a simplified BNF-notation.
>>>>
>>>> The generated parser is based on AntLR and creates a (dynamic) EMF
>>>> model. The generated text editor provides syntax highlighting,
>>>> syntax checking, constraint checking and an outline view.
>>>>
>>>> I've written a blog entry showing some more details (and
>>>> screenshots):
>>>> http://effi-blog.blogspot.com/2006/04/textual-dsl-framework- xtext.html
>>>>
>>>> The xText framework was also a part of our session at "Eclipse Forum
>>>> Europe", which shows the integration of EMF and GMF with oAW (and
>>>> xText). Some of you may find that interesting, too.
>>>> The slides can be found here:
>>>> http://www.eclipse.org/gmt/oaw/doc/s10_gmfAndOAW_slides.pdf
>>>>
>>>> We are very interested in sharing ideas about textual DSL design and
>>>> would be happy if we could contribute some ideas or code to a
>>>> possible TMF (Textual Modelling Framework) if there are plans to
>>>> create such a project.
>>>>
>>>>
>>>> best regards,
>>>> Sven


--
Need professional support on Xtext or Xtend?
Mail to: xtext (at) itemis.com
Twitter : @svenefftinge
Blog : blog.efftinge.de
Re: Textual Modelling Framework? [message #578173 is a reply to message #376408] Sat, 24 June 2006 01:01 Go to previous message
David Williams is currently offline David Williams
Messages: 696
Registered: July 2009
Senior Member
On Tue, 23 May 2006 11:05:18 -0400, Sven Efftinge <sven@efftinge.de> wrote:


>
> We are very interested in sharing ideas about textual DSL design

This sounds like it must be obvious to others on this list, but
what exactly is the advantage of using a "new language" to
describe a DSL instead of, let's say, using XML?
Re: Textual Modelling Framework? [message #578193 is a reply to message #377219] Sat, 24 June 2006 01:55 Go to previous message
Richard Gronback is currently offline Richard Gronback
Messages: 605
Registered: July 2009
Senior Member
Human usability is the primary motivation, I'd say. It's not about creating
a "new language," but providing a concrete syntax for a language that is
more "human friendly" than XML.


On 6/23/06 9:01 PM, in article op.tbmk4agaac05ss@dmw2t23.raleigh.ibm.com,
"David Williams" <david_williams@us.ibm.com> wrote:

> On Tue, 23 May 2006 11:05:18 -0400, Sven Efftinge <sven@efftinge.de> wrote:
>
>
>>
>> We are very interested in sharing ideas about textual DSL design
>
> This sounds like it must be obvious to others on this list, but
> what exactly is the advantage of using a "new language" to
> describe a DSL instead of, let's say, using XML?
>
>
Previous Topic:New PMC Member
Next Topic:Project Scope and UML2/DSL code generation
Goto Forum:
  


Current Time: Fri Oct 31 18:44:30 GMT 2014

Powered by FUDForum. Page generated in 0.03228 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software