Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » two file extenstions with slightly different grammar(Do I need two Xtext projects for two file extenstions with slightly different grammar)
two file extenstions with slightly different grammar [message #693312] Wed, 06 July 2011 08:48 Go to next message
alex.ren2006 is currently offline alex.ren2006Friend
Messages: 46
Registered: June 2011
Member
Hi all,

I am current working on building an IDE for language ATS (www.ats-lang.org). Due to the nature of ATS, my IDE has to deal with two different file extensions: .sats and .dats. .sats file is for function declaration and .dats is for function implementation. It's also legal to write function declaration in .dats file. But the grammar is a little different.

For the clarity of the description, I make up the following dummy grammar.

A valid .sats would look like the following:
int plus1 (int x)


A valid .dats would look like the following:
extern plus2 (int x)

implement plus1 (x) = 
{
    return x + 1
}

implement plus2 (x) = 
{
    return x + 2
}


The grammar for .sats would be like the following:
program:
	body+=statement+;

statement:
	m_decl=decl_base
;
decl_base:
	'int' fun_name=funname '(' 'int' arg=argname ')';

funname:
	name=ID;

argname: 
	name=ID;


The grammar for .dats would be like the following:
program:
	body+=statement+;

statement:
	m_decl=decl
	| m_impl=imple;

decl:
	'extern' decl_base;

decl_base:
	'int' fun_name=funname '(' 'int' arg=argname ')';

funname:
	name=ID;

argname: 
	name=ID;


imple:
	'implement' fun_name=[funname] '(' arg_name=[argname] ')'
	'=' '{' 'return' arg_name1=[argname] '+' INT '}';


I could build two different Xtext projects for .sats and .dats. But since I am going to add cross-referece support to the IDE later (Jump from function declaration in .sats file to implementation in .dats file or vice versa.), what's the good solution in Xtext for this kind of problem? Any suggestion is appreciated. Thanks a lot.
Re: two file extenstions with slightly different grammar [message #693369 is a reply to message #693312] Wed, 06 July 2011 10:25 Go to previous messageGo to next message
Jan Koehnlein is currently offline Jan KoehnleinFriend
Messages: 760
Registered: July 2009
Location: Hamburg
Senior Member
I'd always build separate languages to avoid gnerator conflicts.

As long as one language is a subset of the other, let the second one
inherit from the first. If there's a common subset only, write third
language and let the other two inherit from that.

Cross-language references work OOTB in Xtext.

Am 06.07.11 10:48, schrieb forums-noreply@eclipse.org:
> Hi all,
>
> I am current working on building an IDE for language ATS
> (www.ats-lang.org). Due to the nature of ATS, my IDE has to deal with
> two different file extensions: .sats and .dats. .sats file is for
> function declaration and .dats is for function implementation. It's also
> legal to write function declaration in .dats file. But the grammar is a
> little different.
>
> For the clarity of the description, I make up the following dummy grammar.
>
> A valid .sats would look like the following:
> int plus1 (int x)
>
> A valid .dats would look like the following:
> extern plus2 (int x)
>
> implement plus1 (x) = {
> return x + 1
> }
>
> implement plus2 (x) = {
> return x + 2
> }
>
> The grammar for .sats would be like the following:
> program:
> body+=statement+;
>
> statement:
> m_decl=decl_base
> ;
> decl_base:
> 'int' fun_name=funname '(' 'int' arg=argname ')';
>
> funname:
> name=ID;
>
> argname: name=ID;
>
> The grammar for .dats would be like the following:
> program:
> body+=statement+;
>
> statement:
> m_decl=decl
> | m_impl=imple;
>
> decl:
> 'extern' decl_base;
>
> decl_base:
> 'int' fun_name=funname '(' 'int' arg=argname ')';
>
> funname:
> name=ID;
>
> argname: name=ID;
>
>
> imple:
> 'implement' fun_name=[funname] '(' arg_name=[argname] ')'
> '=' '{' 'return' arg_name1=[argname] '+' INT '}';
>
> I could build two different Xtext projects for .sats and .dats. But
> since I am going to add cross-referece support to the IDE later (Jump
> from function declaration in .sats file to implementation in .dats file
> or vice versa.), what's the good solution in Xtext for this kind of
> problem? Any suggestion is appreciated. Thanks a lot.
>


--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com


---
Get professional support from the Xtext committers at www.typefox.io
Re: two file extenstions with slightly different grammar [message #693408 is a reply to message #693369] Wed, 06 July 2011 12:20 Go to previous messageGo to next message
alex.ren2006 is currently offline alex.ren2006Friend
Messages: 46
Registered: June 2011
Member
Do you mean writing three grammar files like the following:

file "MyDslCommon.xtext"
grammar org.xtext.example.mydsl.MyDslCommon with org.eclipse.xtext.common.Terminals


file "MyDslA.xtext"
grammar org.xtext.example.mydsl.MyDslA with org.xtext.example.mydsl.MyDslCommon


file "MyDslB.xtext"
grammar org.xtext.example.mydsl.MyDslB with org.xtext.example.mydsl.MyDslCommon


So my whole workspace should have 6 projects, 3 for MyDslA, 3 for MyDslB. Is my understanding correct?
Re: two file extenstions with slightly different grammar [message #693482 is a reply to message #693408] Wed, 06 July 2011 14:47 Go to previous messageGo to next message
alex.ren2006 is currently offline alex.ren2006Friend
Messages: 46
Registered: June 2011
Member
What is the proper way?
Re: two file extenstions with slightly different grammar [message #693484 is a reply to message #693369] Wed, 06 July 2011 14:48 Go to previous messageGo to next message
alex.ren2006 is currently offline alex.ren2006Friend
Messages: 46
Registered: June 2011
Member
I tried to devide grammar into 3 parts. Here's what I did. I create 9 projects for 3 grammars.

In project "org.xtext.example.mydsl", I have the grammar file "MyDsl.xtext" with the following content
grammar org.xtext.example.mydsl.MyDsl with org.eclipse.xtext.common.Terminals


generate myDsl "http://www.xtext.org/example/mydsl/MyDsl"


decl_base:
	'int' fun_name=funname '(' 'int' arg=argname ')';

funname:
	name=ID;

argname: 
	name=ID;


In project "org.xtext.example.mydslA", I have the grammar file "MyDslA.xtext" with the following content
grammar org.xtext.example.mydsl.MyDslA with org.xtext.example.mydsl.MyDsl

import "http://www.xtext.org/example/mydsl/MyDsl"

generate myDslA "http://www.xtext.org/example/mydsl/MyDslA"



program:
	body+=statement+;

statement:
	m_decl=decl
	| m_impl=imple;

decl:
	'extern' m_base=decl_base;

imple:
	'implement' fun_name=[funname] '(' arg_name=[argname] ')'
	'=' '{' 'return' arg_name1=[argname] '+' '3' '}';
 


In project "org.xtext.example.mydslB", I have the grammar file "MyDslB.xtext" with the following content
grammar org.xtext.example.mydsl.MyDslB with org.xtext.example.mydsl.MyDsl

generate myDslB "http://www.xtext.org/example/mydsl/MyDslB"


program:  
	m_dec+=decl_base+
;


To stop the xtext editor from generating error, I also add the "org.xtext.example.mydsl/src" to the class folder of the projects "org.xtext.example.mydslA" and "org.xtext.example.mydslB".

There is no problem of running "GenerateMyDsl.mwe2" in project "org.xtext.example.mydsl". But running "GenerateMyDslA.mwe2" never stops at all. The console shows the following message
0    [main] INFO  lipse.emf.mwe.utils.StandaloneSetup  - Registering platform uri 'G:\WORKSPACE\Xtext\workspace_old'
1157 [main] INFO  ipse.emf.mwe.utils.DirectoryCleaner  - Cleaning G:\WORKSPACE\Xtext\workspace_old\org.xtext.example.mydslA\..\org.xtext.example.mydslA\src-gen
1188 [main] INFO  ipse.emf.mwe.utils.DirectoryCleaner  - Cleaning G:\WORKSPACE\Xtext\workspace_old\org.xtext.example.mydslA\..\org.xtext.example.mydslA.ui\src-gen
1313 [main] INFO  ipse.xtext.generator.LanguageConfig  - generating infrastructure for org.xtext.example.mydsl.MyDslA with fragments : ImplicitRuntimeFragment, ImplicitUiFragment, GrammarAccessFragment, EcoreGeneratorFragment, SerializerFragment, ResourceFactoryFragment, XtextAntlrGeneratorFragment, JavaValidatorFragment, ImportNamespacesScopingFragment, QualifiedNamesFragment, BuilderIntegrationFragment, GeneratorFragment, FormatterFragment, LabelProviderFragment, OutlineTreeProviderFragment, QuickOutlineFragment, QuickfixProviderFragment, JavaBasedContentAssistFragment, XtextAntlrUiGeneratorFragment, Junit4Fragment, TypesGeneratorFragment, XbaseGeneratorFragment, CodetemplatesGeneratorFragment, RefactorElementNameFragment, CompareFragment


It seems something is blocked since the CPU usage is very low.

I checked the manual of xtext and add the following to the mwe2 file
            fragment = ecore.EcoreGeneratorFragment {
             referencedGenModels = "
               platform:/resource/org.xtext.example.mydsl/src-gen/org/xtext/example/mydsl/MyDsl.genmodel

             "


Then I can run "GenerateMyDslA.mwe2" successfully with the following message
0    [main] INFO  lipse.emf.mwe.utils.StandaloneSetup  - Registering platform uri 'G:\WORKSPACE\Xtext\workspace_old'
1078 [main] WARN  erator.ecore.EcoreGeneratorFragment  - The property 'referencedGenModels' is deprecated. Please use 'StandaloneSetup.registerGenModelFile' instead.
1109 [main] INFO  ipse.emf.mwe.utils.DirectoryCleaner  - Cleaning G:\WORKSPACE\Xtext\workspace_old\org.xtext.example.mydslA\..\org.xtext.example.mydslA\src-gen
1109 [main] INFO  ipse.emf.mwe.utils.DirectoryCleaner  - Cleaning G:\WORKSPACE\Xtext\workspace_old\org.xtext.example.mydslA\..\org.xtext.example.mydslA.ui\src-gen
1234 [main] INFO  ipse.xtext.generator.LanguageConfig  - generating infrastructure for org.xtext.example.mydsl.MyDslA with fragments : ImplicitRuntimeFragment, ImplicitUiFragment, GrammarAccessFragment, EcoreGeneratorFragment, SerializerFragment, ResourceFactoryFragment, XtextAntlrGeneratorFragment, JavaValidatorFragment, ImportNamespacesScopingFragment, QualifiedNamesFragment, BuilderIntegrationFragment, GeneratorFragment, FormatterFragment, LabelProviderFragment, OutlineTreeProviderFragment, QuickOutlineFragment, QuickfixProviderFragment, JavaBasedContentAssistFragment, XtextAntlrUiGeneratorFragment, Junit4Fragment, TypesGeneratorFragment, XbaseGeneratorFragment, CodetemplatesGeneratorFragment, RefactorElementNameFragment, CompareFragment
2468 [main] INFO  clipse.emf.mwe.utils.GenModelHelper  - Registered GenModel 'http://www.xtext.org/example/mydsl/MyDsl' from 'platform:/resource/org.xtext.example.mydsl/src-gen/org/xtext/example/mydsl/MyDsl.genmodel'
2500 [main] INFO  clipse.emf.mwe.utils.GenModelHelper  - Registered GenModel 'http://www.xtext.org/example/mydsl/MyDslA' from 'file:/G:/WORKSPACE/Xtext/workspace_old/org.xtext.example.mydslA/src-gen/org/xtext/example/mydsl/MyDslA.genmodel'
[b]2500 [main] [color=crimson]WARN  clipse.emf.mwe.utils.GenModelHelper  - There is already a GenModel registered for NamespaceURI 'http://www.xtext.org/example/mydsl/MyDsl'. It will be overwritten from 'platform:/resource/org.xtext.example.mydsl/src-gen/org/xtext/example/mydsl/MyDsl.genmodel' [/color][/b]to 'file:/G:/WORKSPACE/Xtext/workspace_old/org.xtext.example.mydsl/src-gen/org/xtext/example/mydsl/MyDsl.genmodel'
2500 [main] INFO  clipse.emf.mwe.utils.GenModelHelper  - Registered GenModel 'http://www.xtext.org/example/mydsl/MyDsl' from 'file:/G:/WORKSPACE/Xtext/workspace_old/org.xtext.example.mydsl/src-gen/org/xtext/example/mydsl/MyDsl.genmodel'
4562 [main] INFO  or.validation.JavaValidatorFragment  - generating Java-based EValidator API
5046 [main] INFO  text.generator.junit.Junit4Fragment  - generating Junit4 Test support classes
5078 [main] INFO  text.generator.junit.Junit4Fragment  - generating Compare Framework infrastructure
5218 [main] INFO  .emf.mwe2.runtime.workflow.Workflow  - Done.


This time the generated Java code in src-gen folders has errors (e.g.
org.xtext.example.mydsl.MyDslStandaloneSetup cannot be resolved to a type
).

I am not familiar with the ecore, genmodel stuff. What's the proper procedure for importing another grammar in a xtext project? Thanks a lot.
Re: two file extenstions with slightly different grammar [message #693626 is a reply to message #693484] Wed, 06 July 2011 22:43 Go to previous messageGo to next message
alex.ren2006 is currently offline alex.ren2006Friend
Messages: 46
Registered: June 2011
Member
After reading posts about this issue and dealing with all kinds of errors, I think I should just build two totally different languages at this moment which seems the easiest way. What concerns me is whether cross-reference is still possible between two totally different languages. Thank you for your advice.
Re: two file extenstions with slightly different grammar [message #693730 is a reply to message #693626] Thu, 07 July 2011 05:40 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13612
Registered: July 2009
Senior Member
Hi,

i guess the basic problem is the dep between the plugins:
use eclipse plugin deps (manifest.mf file) to to the depenfencies stuff.
(from mydsla to mydsl, from mydsla.ui to mydsl and mydsl.ui)

you may have to add some additional package exports the the manifests of mydsl too
(e.g. contentassist from mydsl.ui)

~Christian


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: two file extenstions with slightly different grammar [message #693776 is a reply to message #693408] Thu, 07 July 2011 07:43 Go to previous messageGo to next message
Jan Koehnlein is currently offline Jan KoehnleinFriend
Messages: 760
Registered: July 2009
Location: Hamburg
Senior Member
yes, that's what I'd propose.

Am 06.07.11 14:20, schrieb forums-noreply@eclipse.org:
> Do you mean writing three grammar files like the following:
>
> file "MyDslCommon.xtext"
> grammar org.xtext.example.mydsl.MyDslCommon with
> org.eclipse.xtext.common.Terminals
>
> file "MyDslA.xtext"
> grammar org.xtext.example.mydsl.MyDslA with
> org.xtext.example.mydsl.MyDslCommon
>
> file "MyDslB.xtext"
> grammar org.xtext.example.mydsl.MyDslB with
> org.xtext.example.mydsl.MyDslCommon
>
> So my whole workspace should have 6 projects, 3 for MyDslA, 3 for
> MyDslB. Is my understanding correct?


--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com


---
Get professional support from the Xtext committers at www.typefox.io
Re: two file extenstions with slightly different grammar [message #693826 is a reply to message #693730] Thu, 07 July 2011 09:22 Go to previous message
alex.ren2006 is currently offline alex.ren2006Friend
Messages: 46
Registered: June 2011
Member
Thank you so much, Christian. It works like breeze.

Honestly speaking, at first I don't understand what you are talking about. I've never done any serious projects in eclipse, let alone plug-in development. But after playing with eclipse for a while I think I get the point from your words.

I write down the whole process using the following example. I have two grammars: commonLang and branchLangA. The latter one depends on the previous one. Also I have to import the ecore model (I am still not very clear about this concept) into grammar branchLangA since I want to use cross-reference.

BTW: I am currently using xtext 2 which seems much better than xtext 1.

1. Create Xtext project
Project name: commonlang
Language name: org.CommonLang
Extensions: comlang


2. The content of CommonLang.xtext goes as follows.
=================
grammar org.CommonLang with org.eclipse.xtext.common.Terminals

generate commonLang "http://www.CommonLang.org"

decl_base:
	'int' fun_name=funname '(' 'int' arg=argname ')';

funname:
	name=ID;

argname: 
	name=ID;
=================


3. Run "GenerateCommonLang.mwe2"


4. Create Xtext project
Project name: branchlanga
Language name: org.BranchLangA
Extensions: brlanga


5. The content of BranchLangA.xtext goes as follows.
=================
grammar org.BranchLangA with org.CommonLang

generate branchLangA "http://www.BranchLangA.org"

import "http://www.CommonLang.org"

program:
	body+=statement+;

statement:
	m_decl=decl
	| m_impl=imple; 

decl:
	'extern' m_base=decl_base;

imple:
	'implement' fun_name=[funname] '(' arg_name=[argname] ')'
	'=' '{' 'return' arg_name1=[argname] '+' '3' '}';
=================


6. Open META-INF->MANIFEST.MF of project branchlanga, click "Dependencies" in "Plug-in Content", and add commlang to the "Required Plug-ins".


7. Modify "GenerateBranchLangA.mwe2", change
==================
            // generates Java API for the generated EPackages
            fragment = ecore.EcoreGeneratorFragment {
            // referencedGenModels = "
            //  platform:/resource/org.eclipse.xtext.xbase/model/Xbase.genmodel,
            //  
platform:/resource/org.eclipse.xtext.common.types/model/JavaVMTypes.genmodel
// "
}
==================
to
==================
            // generates Java API for the generated EPackages
            fragment = ecore.EcoreGeneratorFragment {
             referencedGenModels = "
              platform:/resource/commonlang/src-gen/org/CommonLang.genmodel
             "
            }
=================


8. Run "GenerateBranchLangA.mwe2"


9. At this moment the eclipse editor shows errors in the project "branchlanga", e.g. "BranchLangAScopeProvider.java". Open the file and move the mouse over the error. The popup says as follows
================
Access restriction: The type CommonLangScopeProvider is not accessible due to restriction on required project commonlang

2 quick fixes available
  Export the 'org.scoping' package from the 'commonlang' plug-in
  Search repositories for 'org.scoping'
================
Just click on "Export the 'org.scoping' package from the 'commonlang' plug-in"

Clicking on popups seems not that official. Later I find out that this can be done by openning
META-INF->MANIFEST.MF of project commonlang, click "Runtime" in "Plug-in Content", and add org.scoping to the "Exported Packages".


10. Open META-INF->MANIFEST.MF of project branchlanga.ui, click "Dependencies" in "Plug-in Content", and add commlang.ui to the "Required Plug-ins".


11. Open META-INF->MANIFEST.MF of project commonlang.ui, click "Runtime" in "Plug-in Content", and add org.ui.contentassist to the "Exported Packages".


12. Right click on project branchlanga, choose "Run as" -> "Eclipse Application".
Then I can create file yy.brlanga, and type in code like follows
================
extern int ff (int x)
implement  ff(x) =
{
	return x + 3
}
================

It seems that MANIFEST.MF is much more powerful than classpath or project dependency. (I tried setting classpath and project dependency in "Java Build Path" last night. But it didn't work.) I would like to know more about the machenism beneath. Any suggestion for good book on Java plug-in?

By using xtext, I am befuddled by terms like xtend, xbase, ecore, EMF, mwe and etc. So far I manage to use xtext based on the instinct of programmer and my limited knowledge of Java and of course help from you guys here (thank you once again). Any suggestion for good books on these topics.

Thanks for your help.


















Previous Topic:automatic imports insert
Next Topic:Is FOREACH depreciated?
Goto Forum:
  


Current Time: Sat Sep 26 23:03:32 GMT 2020

Powered by FUDForum. Page generated in 0.01786 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top