Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » embedded languages
embedded languages [message #897294] Mon, 23 July 2012 14:05 Go to next message
Vlad Dumitrescu is currently offline Vlad DumitrescuFriend
Messages: 430
Registered: July 2009
Location: Gothenburg
Senior Member
Hi!

I have a faint memory that this was discussed before, but can't find a reference. [Excuse: I just got back from vacation]

Is it possible to handle multiple languages in a single file, assuming they are easily separated lexically (for example, in different document partitions)? One example is javadoc-style comments; another would be code embedded in a string.

Doing syntax highlighting is relatively easy, the comment/string/whatever token can be parsed and the result can be used in the semantic highlight operation. What about full integration (completion, cross-references, etc)?

When I saw the EmbeddedEditor class, I thought this could be it, but I suppose it refers to embedding it in another UI widget than an Editor... Theoretically, maybe it could be possible to use it within a part of an Editor, having all UI elements first detect which language support they should use at the current cursor position.

Maybe the amount of work needed would be prohibitive, but on the other hand for not many years ago I would have said the same about something like Xtext Smile

best regards,
Vlad
Re: embedded languages [message #897313 is a reply to message #897294] Mon, 23 July 2012 14:52 Go to previous messageGo to next message
Henrik Lindberg is currently offline Henrik LindbergFriend
Messages: 2509
Registered: July 2009
Senior Member
On 2012-23-07 16:05, Vlad Dumitrescu wrote:
> Hi!
>
> I have a faint memory that this was discussed before, but can't find a
> reference. [Excuse: I just got back from vacation]
>
> Is it possible to handle multiple languages in a single file, assuming
> they are easily separated lexically (for example, in different document
> partitions)? One example is javadoc-style comments; another would be
> code embedded in a string.
>
Possible = yes, but not explicitly supported in any way.

> Doing syntax highlighting is relatively easy, the
> comment/string/whatever token can be parsed and the result can be used
> in the semantic highlight operation. What about full integration
> (completion, cross-references, etc)?

I imagine host language would simply recognize an embedded sequence as
Embedded, and put it in a partition. A second parser for the embedded
language kicks in after model is linked - this validates and links the
second language. ResourceDescription for the host language is extended
to also describe the embedded language. When parsing the embedded
language, it probably needs to understand the imports/visibility in the
host language.

> Maybe the amount of work needed would be prohibitive, but on the other
> hand for not many years ago I would have said the same about something
> like Xtext :)
>
Does not sound too difficult actually...

> best regards,
> Vlad
>
Re: embedded languages [message #897332 is a reply to message #897313] Mon, 23 July 2012 15:43 Go to previous messageGo to next message
Vlad Dumitrescu is currently offline Vlad DumitrescuFriend
Messages: 430
Registered: July 2009
Location: Gothenburg
Senior Member
Hi Henrik and thanks for the answer.

It doesn't sound too difficult indeed, but I think it is more difficult than that Smile

There's a difference between manually adding support for language A to embed snippets in language B or C, but generic support has many hard cases (like for example, even language B could embed language D or, horror!, language A).

In addition, there are some semantic question marks too: for example, language A treats each embedded snippet as standalone, but language B might want to group them together...

From a technical point of view, EmbeddedEditor is a good candidate for a starting point: it can use resources that are not files. If one restricts the embedding to one level, I think that the main missing part is an ISourceViewer that links a document partition and doesn't require an own text widget. Or maybe one could define language B/C in a way that sees the language A parts as whitespace and then the same sourceviewer/text widget can be used? Then we "only" need to be able to connect all the languages to the editor and have logic to detect which language is active at each position...

Well, I am just thinking loud. Maybe there isn't enough interest to make this happen, but it certainly would be very cool! Smile

best regards,
Vlad


Re: embedded languages [message #897359 is a reply to message #897332] Mon, 23 July 2012 18:29 Go to previous messageGo to next message
Sebastian Zarnekow is currently offline Sebastian ZarnekowFriend
Messages: 3108
Registered: July 2009
Senior Member
Vlad, Henrik,

the preferences page for template proposals actually does something
similar. It mixes velocity-like tokens with your language tokens. You
may want to dive into that code to get an idea on how to achieve that.
Unfortunately it is not API nor documented in a meaningful manner but
more the result of a very small pet project that I did two years ago.

Regards,
Sebastian
--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com

Am 23.07.12 17:43, schrieb Vlad Dumitrescu:
> Hi Henrik and thanks for the answer.
>
> It doesn't sound too difficult indeed, but I think it is more difficult
> than that :)
> There's a difference between manually adding support for language A to
> embed snippets in language B or C, but generic support has many hard
> cases (like for example, even language B could embed language D or,
> horror!, language A).
> In addition, there are some semantic question marks too: for example,
> language A treats each embedded snippet as standalone, but language B
> might want to group them together...
>
> From a technical point of view, EmbeddedEditor is a good candidate for
> a starting point: it can use resources that are not files. If one
> restricts the embedding to one level, I think that the main missing part
> is an ISourceViewer that links a document partition and doesn't require
> an own text widget. Or maybe one could define language B/C in a way that
> sees the language A parts as whitespace and then the same
> sourceviewer/text widget can be used? Then we "only" need to be able to
> connect all the languages to the editor and have logic to detect which
> language is active at each position...
>
> Well, I am just thinking loud. Maybe there isn't enough interest to make
> this happen, but it certainly would be very cool! :)
>
> best regards,
> Vlad
>
>
>
Re: embedded languages [message #897429 is a reply to message #897359] Tue, 24 July 2012 06:29 Go to previous messageGo to next message
Vlad Dumitrescu is currently offline Vlad DumitrescuFriend
Messages: 430
Registered: July 2009
Location: Gothenburg
Senior Member
Thank you Sebastian!

So in other words, here's another hidden gem in the Xtext code? Smile

What would it take to bring this issue under review for a possible future inclusion in Xtext as API?

regards,
Vlad
Re: embedded languages [message #897483 is a reply to message #897429] Tue, 24 July 2012 09:17 Go to previous messageGo to next message
Sebastian Zarnekow is currently offline Sebastian ZarnekowFriend
Messages: 3108
Registered: July 2009
Senior Member
Am 24.07.12 08:29, schrieb Vlad Dumitrescu:
> Thank you Sebastian!
>
> So in other words, here's another hidden gem in the Xtext code? :)
>
> What would it take to bring this issue under review for a possible
> future inclusion in Xtext as API?
>

You mean something like a big pile of money?

It's more a question of time and a reasonable number of use cases in
order to make sure we find a suitable abstraction.

Regards,
Sebastian
--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com

> regards,
> Vlad
>
Re: embedded languages [message #897489 is a reply to message #897483] Tue, 24 July 2012 09:29 Go to previous messageGo to next message
Vlad Dumitrescu is currently offline Vlad DumitrescuFriend
Messages: 430
Registered: July 2009
Location: Gothenburg
Senior Member
Sebastian Zarnekow wrote on Tue, 24 July 2012 11:17

> What would it take to bring this issue under review for a possible
> future inclusion in Xtext as API?

You mean something like a big pile of money?


Well, I just got an email telling me that I have inherited 19 million USD from some guy in Nigeria, so why not? Wink

Quote:

It's more a question of time and a reasonable number of use cases in
order to make sure we find a suitable abstraction.


Yes, of course. And if there was some spare time, I would rather see first mode detailed docs for the current APIs. This is something I intend to try to contribute back, but I will have first to develop a better understanding of it.

regards,
Vlad

Re: embedded languages [message #897510 is a reply to message #897489] Tue, 24 July 2012 10:37 Go to previous message
Tim Geisler is currently offline Tim GeislerFriend
Messages: 47
Registered: July 2009
Member
I also have a use case.

In VClipse (www.vclipse.org) we implemented DSLs for describing product models in SAP Variant Configuration. This includes descriptions of objects and their relationships (in a language invented by us) and dependency code (in languages invented by SAP). Our first approach was to put all languages into one. However, this has some severe drawbacks, since even the lexical syntax is different (different keywords, keywords in one language might be identifiers in the other, different comment syntax, ...). In the last time we split these languages into several ones and separated the information into several files. So we do not embed anymore.

However, there is one case of dependency code which I really like to have embedded in the language for describing objects: there might be some simple code objects attached to various objects which do not have a global name in SAP and for which I do not want to have a separate file. This is like having short JavaScript code snippets as values of HTML attributes (this is yet another use case).

Unfortunately, we do not have money to invest in such a development...
Previous Topic:Regular Expression and newline
Next Topic:Reading the .mydsl file
Goto Forum:
  


Current Time: Thu Jul 09 00:01:13 GMT 2020

Powered by FUDForum. Page generated in 0.01967 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top