Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Implementing dynamic keywords
Implementing dynamic keywords [message #901505] Mon, 13 August 2012 08:05 Go to next message
Boris Brodski is currently offline Boris BrodskiFriend
Messages: 112
Registered: July 2009
Senior Member
Hello Xtext team!

Is it possible to implement dynamic keywords with the Xtext?

Here is the background: The Cucumber framework (cukes.info or cucumber-jvm) uses spoken language (business readable domain specific language) to define test scenarios. For example:

Feature: Addition
  In order to avoid silly mistakes
  As a math idiot 
  I want to be told the sum of two numbers

  Scenario: Add two numbers
    Given I visit the calculator page
    And I fill in '50' for 'first'
    And I fill in '70' for 'Second'
    When I press 'Add'
    Then I should see 'Answer: 120'


Here is very important, that the keywords (Feature, Scenario, Given, When, Then, ...) can be translated to any other language. Here is an Norwegian (no) example:

# language: no
Egenskap: Summering
  For å unngå at firmaet går konkurs
  Må regnskapsførerere bruke en regnemaskin for å legge sammen tall

  Scenario: to tall
    Gitt at jeg har tastet inn 5
    Og at jeg har tastet inn 7
    Når jeg summerer
    Så skal resultatet være 12

  Scenario: tre tall
    Gitt at jeg har tastet inn 5
    Og at jeg har tastet inn 7
    Og at jeg har tastet inn 1
    Når jeg summerer
    Så skal resultatet være 13


As you can see, the first line sets the language and changes all the keywords!
# language: no

("no" is not the counterpart of the "yes", but the language: no - Norwegian)

How this kind of behavior can be implemented with Xtext?

Thank you very much!
Re: Implementing dynamic keywords [message #901538 is a reply to message #901505] Mon, 13 August 2012 09:47 Go to previous messageGo to next message
Vlad Dumitrescu is currently offline Vlad DumitrescuFriend
Messages: 431
Registered: July 2009
Location: Gothenburg
Senior Member
Hi!

I'm not part of the team, but maybe I can help.

If your translations can be hardcoded, then you can just define
Feature: 'Feature'|'Egenskap'|....;


But somehow I don't think that is a reasonable assumption. Even if you could know all languages beforehand, it can still happen that keywords will clash in different languages, so you will need a custom lexer.

The Xtext grammar will refer to the terminal Feature and the lexer will generate the equivalent of
Feature: 'Feature';
or
Feature:'Egenskap';
depending on the actual language.

If there is a better way, I would be interested to know it too.

best regards,
Vlad


Re: Implementing dynamic keywords [message #901545 is a reply to message #901538] Mon, 13 August 2012 10:36 Go to previous messageGo to next message
Boris Brodski is currently offline Boris BrodskiFriend
Messages: 112
Registered: July 2009
Senior Member
Hello Vlad,


thanks for the reply. I agree with you about your first suggestion. But this is still a solution, if nothing else helps.

The lexer should do this actually, you are right. The question is, how to reuse the lexer produced by the Xtext and override
the keyword matching?
Re: Implementing dynamic keywords [message #901547 is a reply to message #901545] Mon, 13 August 2012 10:41 Go to previous messageGo to next message
Vlad Dumitrescu is currently offline Vlad DumitrescuFriend
Messages: 431
Registered: July 2009
Location: Gothenburg
Senior Member
Hi,

From what I know, you can't just reuse the generated lexer. You have to write one from scratch, possibly using ANTLR which is already used by Xtext. I only did that once and just to check that it works, so I can't give you any technical details, sorry.
Re: Implementing dynamic keywords [message #901575 is a reply to message #901547] Mon, 13 August 2012 13:31 Go to previous messageGo to next message
Henrik Lindberg is currently offline Henrik LindbergFriend
Messages: 2509
Registered: July 2009
Senior Member
On 2012-13-08 12:41, Vlad Dumitrescu wrote:
> Hi,
>
> From what I know, you can't just reuse the generated lexer. You have to
> write one from scratch, possibly using ANTLR which is already used by
> Xtext. I only did that once and just to check that it works, so I can't
> give you any technical details, sorry.
>

Xtext supports an external lexer - it is easily configured in the mwe
workflow.

I am using an external lexer in cloudsmith/geppetto @ github so you can
see an example of this working there. Note that there are two generated
lexers you need to bind to your lexer (one via an adapter). You should
be able to find-out by looking at my code.

Hope that helps.
- henrik
Re: Implementing dynamic keywords [message #904375 is a reply to message #901575] Tue, 28 August 2012 10:44 Go to previous messageGo to next message
Boris Brodski is currently offline Boris BrodskiFriend
Messages: 112
Registered: July 2009
Senior Member
Thank you!

This was indeed very easy. The "normal" parser and syntax highlighting work now.

Do you know, how get also the code completion to work? I still see the old keywords there.
I think, there should be some special method on the lexer, that provides allowed tokens at the
specified position in text. Or it could be sufficient to override the "token index to name" mapping
or something like that.
Re: Implementing dynamic keywords [message #904454 is a reply to message #904375] Tue, 28 August 2012 12:55 Go to previous message
Henrik Lindberg is currently offline Henrik LindbergFriend
Messages: 2509
Registered: July 2009
Senior Member
On 2012-28-08 12:44, Boris Brodski wrote:
> Thank you!
>
> This was indeed very easy. The "normal" parser and syntax highlighting
> work now.
>
> Do you know, how get also the code completion to work? I still see the
> old keywords there.
> I think, there should be some special method on the lexer, that provides
> allowed tokens at the
> specified position in text. Or it could be sufficient to override the
> "token index to name" mapping
> or something like that.
>
No idea. I did not have to do that so never looked into that. I suppose
it is generated or is driven by the grammar.

- henrik
Previous Topic:testing xbase language referring to another xbase resource
Next Topic:How to heck if token is present
Goto Forum:
  


Current Time: Tue Apr 23 08:44:03 GMT 2024

Powered by FUDForum. Page generated in 0.03630 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top