Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Case insensitivity doesn't work with terminal rules
Case insensitivity doesn't work with terminal rules [message #1707078] Wed, 02 September 2015 06:38 Go to next message
Asko Uronen is currently offline Asko UronenFriend
Messages: 4
Registered: September 2015
Junior Member
I'm having problems with the case insensitivity feature in Xtext 2.8.3. The feature seems to work with the parser rules but terminals rules are still case sensitive. I have created a simplified grammar in order to demonstrate the problem:
grammar org.xtext.example.mydsl.MyDsl with org.eclipse.xtext.common.Terminals

generate myDsl "<Links are unallowed in eclipse.org sites>"

Model:
	greetings+=(Greeting|Bye)*;

terminal hello: 'Hello';
Greeting: hello name=ID '!';
Bye: 'Bye' name=ID '!';


mwe2-file is presented below:
module org.xtext.example.mydsl.GenerateMyDsl

import org.eclipse.emf.mwe.utils.*
import org.eclipse.xtext.generator.*
import org.eclipse.xtext.ui.generator.*

var grammarURI = "classpath:/org/xtext/example/mydsl/MyDsl.xtext"
var fileExtensions = "mydsl"
var projectName = "org.xtext.example.mydsl"
var runtimeProject = "../${projectName}"
var generateXtendStub = true
var encoding = "UTF-8"

Workflow {
    bean = StandaloneSetup {
    	scanClassPath = true
    	platformUri = "${runtimeProject}/.."
    	// The following two lines can be removed, if Xbase is not used.
    	registerGeneratedEPackage = "org.eclipse.xtext.xbase.XbasePackage"
    	registerGenModelFile = "platform:/resource/org.eclipse.xtext.xbase/model/Xbase.genmodel"
    }
    
    component = DirectoryCleaner {
    	directory = "${runtimeProject}/src-gen"
    }
    
    component = DirectoryCleaner {
    	directory = "${runtimeProject}/model/generated"
    }
    
    component = DirectoryCleaner {
    	directory = "${runtimeProject}.ui/src-gen"
    }
    
    component = DirectoryCleaner {
    	directory = "${runtimeProject}.tests/src-gen"
    }
    
    component = Generator {
    	pathRtProject = runtimeProject
    	pathUiProject = "${runtimeProject}.ui"
    	pathTestProject = "${runtimeProject}.tests"
    	projectNameRt = projectName
    	projectNameUi = "${projectName}.ui"
    	encoding = encoding
    	language = auto-inject {
    		uri = grammarURI
    
    		// Java API to access grammar elements (required by several other fragments)
    		fragment = grammarAccess.GrammarAccessFragment auto-inject {}
    
    		// generates Java API for the generated EPackages
    		fragment = ecore.EMFGeneratorFragment auto-inject {}
    
    		// the old serialization component
    		// fragment = parseTreeConstructor.ParseTreeConstructorFragment auto-inject {}    
    
    		// serializer 2.0
    		fragment = serializer.SerializerFragment auto-inject {
    			generateStub = false
    		}
    
    		// a custom ResourceFactory for use with EMF
    		fragment = resourceFactory.ResourceFactoryFragment auto-inject {}
    
    		// The antlr parser generator fragment.
    		//fragment = parser.antlr.XtextAntlrGeneratorFragment auto-inject {
    		fragment = parser.antlr.ex.rt.AntlrGeneratorFragment {
    		  options = {
    		      ignoreCase = true //FIXME: no effect
    		//      backtrack = true
    		  }
    		}
    
    		// Xtend-based API for validation
    		fragment = validation.ValidatorFragment auto-inject {
    		//    composedCheck = "org.eclipse.xtext.validation.ImportUriValidator"
    		//    composedCheck = "org.eclipse.xtext.validation.NamesAreUniqueValidator"
    		}
    
    		// old scoping and exporting API
    		// fragment = scoping.ImportURIScopingFragment auto-inject {}
    		// fragment = exporting.SimpleNamesFragment auto-inject {}
    
    		// scoping and exporting API
    		//fragment = scoping.ImportNamespacesScopingFragment auto-inject {}
    		fragment = scoping.ImportNamespacesScopingFragment {
  	 			ignoreCase = true
    		}
    		fragment = exporting.QualifiedNamesFragment auto-inject {}
    		fragment = builder.BuilderIntegrationFragment auto-inject {}
    
    		// generator API
    		fragment = generator.GeneratorFragment auto-inject {}
    
    		// formatter API
    		fragment = formatting.FormatterFragment auto-inject {}
    
    		// labeling API
    		fragment = labeling.LabelProviderFragment auto-inject {}
    
    		// outline API
    		fragment = outline.OutlineTreeProviderFragment auto-inject {}
    		fragment = outline.QuickOutlineFragment auto-inject {}
    
    		// quickfix API
    		fragment = quickfix.QuickfixProviderFragment auto-inject {}
    
    		// content assist API
    		fragment = contentAssist.ContentAssistFragment auto-inject {}
    
    		// generates a more lightweight Antlr parser and lexer tailored for content assist
    		//fragment = parser.antlr.XtextAntlrUiGeneratorFragment auto-inject {}
    		fragment = parser.antlr.ex.ca.ContentAssistParserGeneratorFragment {
    			options = {
    				ignoreCase = true
    			}
    		}
    
    		// generates junit test support classes into Generator#pathTestProject
    		fragment = junit.Junit4Fragment auto-inject {}
    
    		// rename refactoring
    		fragment = refactoring.RefactorElementNameFragment auto-inject {}
    
    		// provides the necessary bindings for java types integration
    		fragment = types.TypesGeneratorFragment auto-inject {}
    
    		// generates the required bindings only if the grammar inherits from Xbase
    		fragment = xbase.XbaseGeneratorFragment auto-inject {}
    		
    		// generates the required bindings only if the grammar inherits from Xtype
    		fragment = xbase.XtypeGeneratorFragment auto-inject {}
    
    		// provides a preference page for template proposals
    		fragment = templates.CodetemplatesGeneratorFragment auto-inject {}
    
    		// provides a compare view
    		fragment = compare.CompareFragment auto-inject {}
    	}
    }
}


Following results were observed:
bye Tom! 
hello John! //missing EOF at 'hello'


Is there a way to implement case insensitive terminal rules? If no then what would be the simplest work-a-around?
Re: Case insensitivity doesn't work with terminal rules [message #1707147 is a reply to message #1707078] Wed, 02 September 2015 14:52 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14665
Registered: July 2009
Senior Member
I Think this is intended.
you should never convert keywords to terminals but just use them as keywords


Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Case insensitivity doesn't work with terminal rules [message #1707195 is a reply to message #1707147] Thu, 03 September 2015 04:11 Go to previous messageGo to next message
Asko Uronen is currently offline Asko UronenFriend
Messages: 4
Registered: September 2015
Junior Member
Thank you for your answer. It seems that indeed if I convert the terminal rule to parser rule the case insensitivity appears to be working.
grammar org.xtext.example.mydsl.MyDsl with org.eclipse.xtext.common.Terminals

generate myDsl "<Links are unallowed in eclipse.org sites>"

Model:
	greetings+=(Greeting|Bye)*;

hello: 'Hello';
Greeting: hello name=ID '!';
Bye: 'Bye' name=ID '!';


However, I fail to see why exactly terminal rules cannot be keywords. The Xtext documentation states:

Terminal Rules

In the first stage called lexing, a sequence of characters (the text input) is transformed
into a sequence of so called tokens. In this context, a token is sort of a strongly typed
part or region of the input sequence. It consists of one or more characters and was
matched by a particular terminal rule or keyword and therefore represents an atomic
symbol.
Re: Case insensitivity doesn't work with terminal rules [message #1707196 is a reply to message #1707195] Thu, 03 September 2015 04:40 Go to previous message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14665
Registered: July 2009
Senior Member
Keywords are terminal rules. I think the case insensivity is done
When the keyword is translated to a terminal. Terminal rules can contain complicated stuff so that such a translation is usually impossible for term. Rules


Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Previous Topic:New formatter API
Next Topic:Generator using external compiler
Goto Forum:
  


Current Time: Tue Apr 23 09:06:48 GMT 2024

Powered by FUDForum. Page generated in 0.03820 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top