Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Parsing Xtext-File with Xtext(Parsing xtext-file with xtext to generate some code for every xtext-grammar is passed)
Parsing Xtext-File with Xtext [message #1697499] Thu, 04 June 2015 12:12 Go to next message
Tamara Machowa is currently offline Tamara MachowaFriend
Messages: 22
Registered: June 2015
Junior Member
In order to generate some code from xtext-grammars, I would like to parse a xtext grammar with xtext. Therefore I could use the grammar from GitHub and create a xtext-grammar that would parse any xtext-file, or I could extend the xtext-editor itself.

Which approach is the better one? And how should it be done propperly?
Re: Parsing Xtext-File with Xtext [message #1697504 is a reply to message #1697499] Thu, 04 June 2015 13:17 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14661
Registered: July 2009
Senior Member
Hi can you elaborate what kind of stuff you want to generate? Depending on your usecase you could simply write a fragment that hooks into the xtext generator in the workflow. Besides this xtext makes the grammar available through XxxDslGrammarAccess.

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Parsing Xtext-File with Xtext [message #1697511 is a reply to message #1697499] Thu, 04 June 2015 13:45 Go to previous messageGo to next message
Tamara Machowa is currently offline Tamara MachowaFriend
Messages: 22
Registered: June 2015
Junior Member
I would like a generator that would generate a SSLR-plugin. It consists of 4 maven
projects and in one of them I have to define the grammar that will be used by SSLR and
later by Sonar. Therefore I need the AST of the grammar-file itself - or something
similar to iterate over all rules and keywords.

How does a fragment look like? Could you give me an example?
Edit: I looked at the ValidatorFragment and I think, that I understand what you mean.
But how should I provide my fragment? Should I create a xtend-project and add it to
the dependencies of the xtext-project I would generate a SSLR-project? How does it work?

[Updated on: Thu, 04 June 2015 13:57]

Report message to a moderator

Re: Parsing Xtext-File with Xtext [message #1697512 is a reply to message #1697511] Thu, 04 June 2015 14:11 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14661
Registered: July 2009
Senior Member
You should provide it as plugin project

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Parsing Xtext-File with Xtext [message #1697516 is a reply to message #1697512] Thu, 04 June 2015 14:26 Go to previous messageGo to next message
Tamara Machowa is currently offline Tamara MachowaFriend
Messages: 22
Registered: June 2015
Junior Member
Is were a documentation which I could consult for upcomming questions?

I figured out, that I should extend the class from Xtend2GeneratorFragment and obviously implement the interfaces IInheriting and IStubGenerating but why I am forced to implement isInheritImplementation, setInheritImplementation, isGenerateStub and setGenerateStub while the ValidatorFragment for instance misses these methods?
Re: Parsing Xtext-File with Xtext [message #1697517 is a reply to message #1697516] Thu, 04 June 2015 14:28 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14661
Registered: July 2009
Senior Member
There is no docs at all. Thus you have to read code. Maybe Format*2* fragment is a goodstarting point. The properties make the fragment configurable from the workflow

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Parsing Xtext-File with Xtext [message #1697522 is a reply to message #1697517] Thu, 04 June 2015 14:48 Go to previous messageGo to next message
Tamara Machowa is currently offline Tamara MachowaFriend
Messages: 22
Registered: June 2015
Junior Member
Do you mean FormatterFragment? Actually I need something that would generate some code, like the ValidationFragment. Do I miss something?
Re: Parsing Xtext-File with Xtext [message #1697524 is a reply to message #1697522] Thu, 04 June 2015 14:57 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14661
Registered: July 2009
Senior Member
Hi,

i mean org.eclipse.xtext.generator.formatting2.Formatter2Fragment that uses FormatterStubGenerator


Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Parsing Xtext-File with Xtext [message #1697528 is a reply to message #1697524] Thu, 04 June 2015 15:21 Go to previous messageGo to next message
Tamara Machowa is currently offline Tamara MachowaFriend
Messages: 22
Registered: June 2015
Junior Member
I found it already by myself Embarrassed

May I ask you another question? What are the methods getGuiceBindingsRt and getGuiceBindingsUi for? And is there a way to build a separate project folder? PLUGIN_RT seems to be the project where the xtext-file is in. So the workspace can be accessed through PLUGIN_IDE?
Re: Parsing Xtext-File with Xtext [message #1697529 is a reply to message #1697528] Thu, 04 June 2015 15:24 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14661
Registered: July 2009
Senior Member
Hi,

these are for the bindings of in yourdsl and yourdsl.ui manifest.mf files.
no there is by default only support for these and the test project.

for and additional project there is nothing ready


Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Parsing Xtext-File with Xtext [message #1697592 is a reply to message #1697529] Fri, 05 June 2015 08:32 Go to previous messageGo to next message
Tamara Machowa is currently offline Tamara MachowaFriend
Messages: 22
Registered: June 2015
Junior Member
Hi,

how could I provide something that would create something in my workspace. If I'm understood it correctly, the abstract class Output defines the functionality to resolve the outletname and create the corresponding path. How could I use this without the outletname?
Re: Parsing Xtext-File with Xtext [message #1697598 is a reply to message #1697592] Fri, 05 June 2015 09:15 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14661
Registered: July 2009
Senior Member
sry you have the read/debug the code for that yourself. it would be easier if you create the project manually and create files only

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Parsing Xtext-File with Xtext [message #1697627 is a reply to message #1697598] Fri, 05 June 2015 13:14 Go to previous messageGo to next message
Tamara Machowa is currently offline Tamara MachowaFriend
Messages: 22
Registered: June 2015
Junior Member
For those, who have a similar use case - here is my current solution:

1) Create a Plugin-Project that would contain the new fragment.

2) Create the Fragment class:

package myCompany.fragments.sslr

import com.google.inject.Inject
import org.eclipse.xtend.lib.annotations.Accessors
import org.eclipse.xtext.Grammar
import org.eclipse.xtext.generator.Generator
import org.eclipse.xtext.generator.Xtend2ExecutionContext
import org.eclipse.xtext.generator.Xtend2GeneratorFragment

class SSLRGeneratorFragment extends Xtend2GeneratorFragment implements IStubLexerless {
	
	SSLRGrammarGenerator grammarGenerator
	
	@Accessors boolean generateLexerless

	@Inject
	def void init(SSLRGrammarGenerator.Service stubGeneratorService, Grammar grammar) {
		this.grammarGenerator = stubGeneratorService.createGenerator(grammar)
	}

	def String cls(Class<?> clazz) {
		clazz.name + ".class"
	}

	override generate(Xtend2ExecutionContext ctx) {
		ctx.writeFile(Generator.SRC_GEN, grammarGenerator.stubFileName, grammarGenerator.generateStubFileContents(generateLexerless))
	}
}


Generator.SRC_GEN will generate the desired File in the src-gen-folder of the xtext-project. Again the question: How could I configure this to wirte to the workspace? Actually I don't come very far with debugging the code Sad

3) Create the Stubgenerator class

package myCompany.fragments.sslr

...

import org.sonar.sslr.grammar.GrammarRuleKey
import org.sonar.sslr.grammar.LexerfulGrammarBuilder
import org.sonar.sslr.grammar.LexerlessGrammarBuilder

import static extension org.eclipse.xtext.GrammarUtil.*

@FinalFieldsConstructor class SSLRGrammarGenerator {
	@Accessors(PUBLIC_GETTER) static class Service {
		@Inject Naming naming

		def SSLRGrammarGenerator createGenerator(Grammar grammar) {
			new SSLRGrammarGenerator(this, grammar)
		}
	}

	val SSLRGrammarGenerator.Service service
	val Grammar grammar
	val Set<AbstractRule> calledRules = Sets.newHashSet()
	val HashMap<String, String> punctuators = Maps.newHashMap()

	def String getStubSimpleName() {
		'''«service.naming.toSimpleName(grammar.name)»Grammar'''
	}

	def String getStubPackageName() {
		'''«service.naming.toPackageName(grammar.name)».grammar'''
	}

	def String getStubQualifiedName() {
		'''«stubPackageName».«stubSimpleName»'''
	}

	def String getStubFileName() {
		'''«service.naming.asPath(getStubQualifiedName)».java'''
	}

	def String getStubSuperClassName() {
		return GrammarRuleKey.name
	}
	
	def setFileHeader(String fileHeader){
		service.naming.fileHeader = fileHeader
	}

        // now we can write some templates to generate the desired input
	def String generateStubFileContents(boolean generateLexerless) {
		val extension file = new JavaEMFFile(grammar.eResource.resourceSet, stubPackageName, service.naming.fileHeader);
		file.imported(GrammarRuleKey)
		if(generateLexerless)
			file.imported(LexerlessGrammarBuilder)
		else
			file.imported(LexerfulGrammarBuilder)

		val grammarBuilder = '''«IF generateLexerless»LexerlessGrammarBuilder«ELSE»LexerfulGrammarBuilder«ENDIF»'''
		val abstractRulesWithoutTerminals = grammar.allRules.filter[!(it instanceof TerminalRule)]
		val usedRulesFinder = new UsedRulesFinder(calledRules)
		usedRulesFinder.compute(grammar)
		val terminalRules = grammar.allTerminalRules.filter[isUsed && !#['ML_COMMENT','SL_COMMENT'].contains(name)]
		val keywords = grammar.allParserRules
							.map[eAllContents.filter(Keyword)]
							.map[toList].filter[!nullOrEmpty].flatten.distinct
		getPunctuators(keywords.map[value].filter[!isAlphanumeric], punctuators)
		
		file.body = '''
			public enum «stubSimpleName» implements «stubSuperClassName.imported» {
				
				«IF !terminalRules.nullOrEmpty»
					// Terminal Rules
					«IF terminalRules.size > 1»
						«terminalRules
							.effect[name=if(name.equals('WS')) 'WHITESPACE' else name]
							.map[name.toUpperCase].join(',\n')»«IF abstractRulesWithoutTerminals.nullOrEmpty && punctuators.empty»;«ELSE»,«ENDIF»
					«ELSEIF abstractRulesWithoutTerminals.nullOrEmpty && punctuators.empty»
						«terminalRules.head.name.toUpperCase»;
					«ELSE»
						«terminalRules.head.name.toUpperCase»,
					«ENDIF»
				«ENDIF»
				«IF !punctuators.empty»
					«punctuators.values.join(',\n')»«IF abstractRulesWithoutTerminals.nullOrEmpty»;«ELSE»,«ENDIF»
				«ENDIF»
				
				«IF !abstractRulesWithoutTerminals.nullOrEmpty»
					// NonTerminal Rules
					«IF abstractRulesWithoutTerminals.size > 1»
						«abstractRulesWithoutTerminals.map[name.toFirstUpper].join(',\n')»;
					«ELSE»
						«abstractRulesWithoutTerminals.head.name.toFirstUpper»;
					«ENDIF»
				«ENDIF»
				
				public static «grammarBuilder» createGrammarBuilder() {
					«grammarBuilder» b = «grammarBuilder».create();
					
					«IF !abstractRulesWithoutTerminals.nullOrEmpty»
						// NonTerminal Rules
						«var i = 0»
						«FOR r: abstractRulesWithoutTerminals»
							«IF i != 0»
								«r.genRule»
							«ELSE»
								«r.genRoot»
							«ENDIF»
							«{i++; null}»
						«ENDFOR»
					«ENDIF»
					
					«IF !terminalRules.nullOrEmpty»
						// Terminal Rules
						b.rule(WHITESPACE).is(b.regexp("\\s*+"));
						«FOR rule: terminalRules.filter[!name.equals('WHITESPACE')]»
							«rule.genRule»
						«ENDFOR»
					«ENDIF»
					
					«IF !punctuators.empty»
						// Punctuators
						«FOR keyword: punctuators.toPairs»
							«keyword.genRule»
						«ENDFOR»
					«ENDIF»
					
					b.setRootRule(«grammar.rules.head.name»);
					
					return b;
				}
			}
		'''
		return file.toString
	}
...


4) Beacuse I wanted an other property to be set in the workflow, I had to implement also this:

package myCompany.fragments.sslr;

public interface IStubLexerless {
	
	boolean isGenerateLexerless();

	void setGenerateLexerless(boolean isGenerateLexerless);
}


5) Add this plugin as a depencies to the xtext-project you want to extend and add the following lines in the workflow:

import myCompany.fragments.*

// SSLR-Fragment
fragment = sslr.SSLRGeneratorFragment auto-inject {
	generateLexerless = true
}


Works like a charm - as long as you don't want to generate a whole project.
Re: Parsing Xtext-File with Xtext [message #1698994 is a reply to message #1697627] Fri, 19 June 2015 11:25 Go to previous message
Tamara Machowa is currently offline Tamara MachowaFriend
Messages: 22
Registered: June 2015
Junior Member
For those, who want to generate files into an other projact than the xtext-projects, you can register other outlets than the normally used in the Generator.

Have a look at the IdeaPluginGenerator
https://github.com/eclipse/xtext/blob/master/intellij/org.eclipse.xtext.idea.generator/src/org/eclipse/xtext/idea/generator/IdeaPluginGenerator.xtend

and the corresponding extension where the needed parts are implemented
https://github.com/eclipse/xtext/blob/master/intellij/org.eclipse.xtext.idea.generator/src/org/eclipse/xtext/idea/generator/parser/antlr/XtextIDEAGeneratorExtensions.xtend

This is still not a generation of a project, but a nice workaround.

[Updated on: Fri, 19 June 2015 11:44]

Report message to a moderator

Previous Topic:IEObjectDocumentationProvider-Labels too small
Next Topic:Conflicting extension parsers
Goto Forum:
  


Current Time: Tue Mar 19 11:05:57 GMT 2024

Powered by FUDForum. Page generated in 0.02457 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top