Eclipse Community Forums: TMF (Xtext) » first and last character truncated in STRING terminals

Home » Modeling » TMF (Xtext) » first and last character truncated in STRING terminals

first and last character truncated in STRING terminals [message #943160]

Sun, 14 October 2012 03:51

Eclipse User

I have redefined the STRING terminal this way

terminal STRING : ('.'|'+'|'('|')'|'a'..'z'|'A'..'Z'|'_'|'0'..'9')*;

because I have to recognize STRING not delimited by " or '

the problem is that, though the generated parser works, it truncates the first and the last character of the recognized string. What am I missing?

Re: first and last character truncated in STRING terminals [message #943165 is a reply to message #943160]

Sun, 14 October 2012 03:58

Eclipse User

Hi you have to change the valueconverter too

Re: first and last character truncated in STRING terminals [message #943172 is a reply to message #943165]

Sun, 14 October 2012 04:07

Eclipse User

Hi Christian,
thank you for your answer.
What is the easiest way to do that, considering that I just need to turn that into an EString?

Re: first and last character truncated in STRING terminals [message #943179 is a reply to message #943172]

Sun, 14 October 2012 04:15

Eclipse User

Hi

then simply do a one to one mapping in the valueconverter.
http://www.eclipse.org/Xtext/documentation.html#valueconverter
(ovveride org.eclipse.xtext.common.services.DefaultTerminalConverters.STRING())

~Christian

Re: first and last character truncated in STRING terminals [message #943187 is a reply to message #943179]

Sun, 14 October 2012 04:27

Eclipse User

I am diving into the DefaultTerminalConverters in order to see how the STRING conversion was made and properly ovverride it. However I see the string conversion is delegated to an injected object (stringValueConverter). Where is that defined ? Xtext's architecture is so well made and so loose coupled that I can't find where things are actually done Smile

Re: first and last character truncated in STRING terminals [message #943188 is a reply to message #943187]

Sun, 14 October 2012 04:28

Eclipse User

Ok My bad I had to check here package org.eclipse.xtext.conversion.impl;
for the StringValueConverter Class
I guess I also need the AbstractLexerBasedConverter ?

Re: first and last character truncated in STRING terminals [message #943192 is a reply to message #943188]

Sun, 14 October 2012 04:31

Eclipse User

simply do something similar to the class org.eclipse.xtext.conversion.impl.STRINGValueConverter

Re: first and last character truncated in STRING terminals [message #943196 is a reply to message #943192]

Sun, 14 October 2012 04:34

Eclipse User

OK now I have a class extending DefaultTerminalConverters and overriding its public IValueConverter<String> STRING() method.

Such method returns an instance of another self made class extending AbstractLexerBasedConverter<String> which basically does the same things of STRINGValueCoverter but it does not substring the original string.

No What do I have to do in the runtime module to make xtext using my own TerminalConverter?

Re: first and last character truncated in STRING terminals [message #943201 is a reply to message #943196]

Sun, 14 October 2012 04:42

Eclipse User

Ok I can answer myself

@Override
public Class<? extends IValueConverterService>
bindIValueConverterService() {
return MyStringValueConverter.class;
}

in the RuntimeModule and it works perfectly

Re: first and last character truncated in STRING terminals [message #943202 is a reply to message #943196]

Sun, 14 October 2012 04:42

Eclipse User

Hi,

have a look at the existing binding and you will know what to do

	public Class<? extends org.eclipse.xtext.conversion.IValueConverterService> bindIValueConverterService() {
		return DefaultTerminalConverters.class;
	}

~Christian

Re: first and last character truncated in STRING terminals [message #943203 is a reply to message #943201]

Sun, 14 October 2012 04:43

Eclipse User

Christian Thank you again for your kind help!

Re: first and last character truncated in STRING terminals [message #943392 is a reply to message #943187]

Sun, 14 October 2012 08:43

Eclipse User

On 2012-14-10 10:27, Andrea Sindico wrote:
> I am diving into the DefaultTerminalConverters in order to see how the
> STRING conversion was made and properly ovverride it. However I see the
> string conversion is delegated to an injected object
> (stringValueConverter). Where is that defined ? Xtext's architecture is
> so well made and so loose coupled that I can't find where things are
> actually done :)
Tip,
If injection is made via interface, look at the source for the interface
- see if it has a ImplementedBy annotation. That is the class that is
used unless there is a binding in the module(s) used to setup your langague.

Secondly, look the module for your language, open type hierarchy and
include all inherited methods. See if you find the class there.

Typically, I find it easier to just debug, set a breakpoint at look at
the class of the injected thing.

Regards
- henrik

Re: first and last character truncated in STRING terminals [message #959384 is a reply to message #943160]

Fri, 26 October 2012 13:24

Eclipse User

Andrea,

I am attempting to do the same thing--have a grammar that supports unquoted STRINGs. As the first step, I am overriding the terminal STRING as you did:

terminal STRING : ('.'|'+'|'('|')'|'a'..'z'|'A'..'Z'|'_'|'0'..'9')*;

But, when I run the MWE2 Workflow, I get this error:

error(208): ../org.xtext.example.definitions/src-gen/org/xtext/example/definitions/parser/antlr/internal/InternalDefinitions.g:243:1: The following token definitions can never be matched because prior tokens match the same input: RULE_INT

I assume you had the same problem, and if so, what did you do?

Thanks!

Re: first and last character truncated in STRING terminals [message #960514 is a reply to message #959384]

Sat, 27 October 2012 10:50

Eclipse User

Please refer to the section on terminal rules in the documentation. Your
STRING rule will basicly consume everything like ID and INTs since it is
defined with a higher priority (because the others are inherited).

Regards,
Sebastian
--
Looking for professional support for Xtext, Xtend or Eclipse Modeling?
Go visit: http://xtext.itemis.com

Am 26.10.12 19:24, schrieb Kokongi Nei:
> Andrea,
>
> I am attempting to do the same thing--have a grammar that supports
> unquoted STRINGs. As the first step, I am overriding the terminal
> STRING as you did:
>
> terminal STRING : ('.'|'+'|'('|')'|'a'..'z'|'A'..'Z'|'_'|'0'..'9')*;
>
> But, when I run the MWE2 Workflow, I get this error:
>
> error(208):
> ../org.xtext.example.definitions/src-gen/org/xtext/example/definitions/parser/antlr/internal/InternalDefinitions.g:243:1:
> The following token definitions can never be matched because prior
> tokens match the same input: RULE_INT
>
> I assume you had the same problem, and if so, what did you do?
>
> Thanks!

Previous Topic:	Issue with inferred model and variable usage
Next Topic:	pre-generate resource descriptions

Goto Forum:

-=] Back to Top [=-

Current Time: Mon Jul 07 22:10:35 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter