Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » XTEXT does not consume terminals like intended
XTEXT does not consume terminals like intended [message #1661491] Mon, 09 March 2015 17:02 Go to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
Hi, Im new to Xtext and I'm trying to get a SPARQL query editor to run.

I have so far "translated" an ebnf grammar for SPARQL into xtext. I'm having trouble with terminal definitions. Eclipse builds the xtext file as intended but when I enter some basic examples into the runtime eclipse editor (started after building the xtext grammar file) i get some confusing errors.

The grammar looks like follows:
grammar org.xtext.Sparql hidden(WS)

import '...' as ecore
generate sparql "..."

//[1]     Query     ::=       Prologue ( SelectQuery | ConstructQuery | DescribeQuery | AskQuery )
Query:
	Prologue (query=SelectQuery | query=ConstructQuery | query=DescribeQuery | query=AskQuery);

	//[2]     Prologue      ::=       BaseDecl? PrefixDecl*
Prologue:
	{Prologue} base=BaseDecl? prefixes+=PrefixDecl*;

	//[3]     BaseDecl      ::=       'BASE' IRI_REF
BaseDecl:
	'BASE' iri=IRI_REF;

	//[4]     PrefixDecl    ::=       'PREFIX' PNAME_NS IRI_REF
PrefixDecl:
	'PREFIX' PNAME_NS iri=IRI_REF;

	//[5]     SelectQuery       ::=       'SELECT' ( 'DISTINCT' | 'REDUCED' )? ( Var+ | '*' ) DatasetClause* WhereClause SolutionModifier
SelectQuery:
	'SELECT' ({SelectDistinctQuery} 'DISTINCT' | {SelectReducedQuery} 'REDUCED')? (projectVars+=Var+ | '*')
	dataSets+=DatasetClause* whereClause=WhereClause solutionModifier=SolutionModifier;

	//[6]     ConstructQuery    ::=       'CONSTRUCT' ConstructTemplate DatasetClause* WhereClause SolutionModifier
ConstructQuery:
	'CONSTRUCT' template=ConstructTemplate dataSets+=DatasetClause* whereClause=WhereClause
	solutionmodifier=SolutionModifier;

	//[7]     DescribeQuery     ::=       'DESCRIBE' ( VarOrIRIref+ | '*' ) DatasetClause* WhereClause? SolutionModifier
DescribeQuery:
	'DESCRIBE' (projectvars+=VarOrIRIref+ | '*') dataSets+=DatasetClause* whereClasue=WhereClause?
	solutionModifier=SolutionModifier;

	//[8]     AskQuery      ::=       'ASK' DatasetClause* WhereClause
AskQuery:
	'ASK' (dataSets+=DatasetClause)* whereClause=WhereClause;

	//[9]     DatasetClause     ::=       'FROM' ( DefaultGraphClause | NamedGraphClause )
DatasetClause:
	'FROM' (DefaultGraphClause | NamedGraphClause);

	//[10]    DefaultGraphClause    ::=       SourceSelector
DefaultGraphClause:
	iri=SourceSelector;

	//[11]    NamedGraphClause      ::=       'NAMED' SourceSelector
NamedGraphClause:
	'NAMED' iri=SourceSelector;

	//[12]    SourceSelector    ::=       IRIref
SourceSelector:
	IRIref;

	//[13]    WhereClause       ::=       'WHERE'? GroupGraphPattern
WhereClause:
	{WhereClause} 'WHERE'? groupGraphpattern=GroupGraphPattern;

	//[14]    SolutionModifier      ::=       OrderClause? LimitOffsetClauses?
SolutionModifier:
	{SolutionModifier} (orderClause=OrderClause)? (limitClause=LimitOffsetClauses)?;

	//[15]    LimitOffsetClauses    ::=       ( LimitClause OffsetClause? | OffsetClause LimitClause? )
LimitOffsetClauses:
	(LimitClause OffsetClause? | OffsetClause LimitClause?);

	//[16]    OrderClause       ::=       'ORDER' 'BY' OrderCondition+
OrderClause:
	'ORDER' 'BY' conditions+=OrderCondition+;

	//[17]    OrderCondition    ::=       ( ( 'ASC' | 'DESC' ) BrackettedExpression )| ( Constraint | Var )
OrderCondition:
	'ASC' ascExpr=BrackettedExpression | 'DESC' descExpr=BrackettedExpression | Constraint | Var;

	//[18]    LimitClause       ::=       'LIMIT' INTEGER
LimitClause:
	'LIMIT' INTEGER;

	//[19]    OffsetClause      ::=       'OFFSET' INTEGER
OffsetClause:
	'OFFSET' INTEGER;

	//[20]    GroupGraphPattern     ::=       '{' TriplesBlock? ( ( GraphPatternNotTriples | Filter ) '.'? TriplesBlock? )* '}'
GroupGraphPattern:
	{GroupGraphPattern}
	'{' (triples+=TriplesBlock)? ((graphPattern+=GraphPatternNotTriples | filter+=Filter) '.'? (triples+=TriplesBlock)?)*
	'}';

	//[21]    TriplesBlock      ::=       TriplesSameSubject ( '.' TriplesBlock? )?
TriplesBlock:
	triples+=TriplesSameSubject ('.' (triples+=TriplesBlock)?)?;

	//[22]    GraphPatternNotTriples    ::=       OptionalGraphPattern | GroupOrUnionGraphPattern | GraphGraphPattern
GraphPatternNotTriples:
	optional=OptionalGraphPattern | union=GroupOrUnionGraphPattern | graph=GraphGraphPattern;

	//[23]    OptionalGraphPattern      ::=       'OPTIONAL' GroupGraphPattern
OptionalGraphPattern:
	'OPTIONAL' GroupGraphPattern;

	//[24]    GraphGraphPattern     ::=       'GRAPH' VarOrIRIref GroupGraphPattern
GraphGraphPattern:
	'GRAPH' varOrIri=VarOrIRIref groupgraph=GroupGraphPattern;

	//[25]    GroupOrUnionGraphPattern      ::=       GroupGraphPattern ( 'UNION' GroupGraphPattern )*
GroupOrUnionGraphPattern:
	unionGrupGraph+=GroupGraphPattern ('UNION' unionGrupGraph+=GroupGraphPattern)*;

	//[26]    Filter    ::=       'FILTER' Constraint
Filter:
	'FILTER' Constraint;

	//[27]    Constraint    ::=       BrackettedExpression | BuiltInCall | FunctionCall
Constraint:
	BrackettedExpression | BuiltInCall | FunctionCall;

	//[28]    FunctionCall      ::=       IRIref ArgList
FunctionCall:
	iri=IRIref argList=ArgList;

	//[29]    ArgList       ::=       ( NIL | '(' Expression ( ',' Expression )* ')' )
ArgList:
	({ArgList} NIL | '(' args+=Expression (',' args+=Expression)* ')');

	//[30]    ConstructTemplate     ::=       '{' ConstructTriples? '}'
ConstructTemplate:
	{ConstructTemplate} '{' (constructTriples=ConstructTriples)? '}';

	//[31]    ConstructTriples      ::=       TriplesSameSubject ( '.' ConstructTriples? )?
ConstructTriples:
	TriplesSameSubject ('.' (constructTriples=ConstructTriples)?)?;

	//[32]    TriplesSameSubject    ::=       VarOrTerm PropertyListNotEmpty | TriplesNode PropertyList
TriplesSameSubject:
	subject=VarOrTerm propertyList=PropertyListNotEmpty | subject=TriplesNode propertyList=PropertyList;

	//[33]    PropertyListNotEmpty      ::=       Verb ObjectList ( ';' ( Verb ObjectList )? )*
PropertyListNotEmpty:
	verbs+=Verb objectLists+=ObjectList (';' (verbs+=Verb objectLists+=ObjectList)?)*;

	//[34]    PropertyList      ::=       PropertyListNotEmpty?
PropertyList:
	{PropertyList} (propertyList=PropertyListNotEmpty)?;

	//[35]    ObjectList    ::=       Object ( ',' Object )*
ObjectList:
	objects+=Object (',' objects+=Object)*;

	//[36]    Object    ::=       GraphNode
Object:
	value=GraphNode;

	//[37]    Verb      ::=       VarOrIRIref | 'a'
Verb:
	VarOrIRIref | 'a' {IRIref};

	//[38]    TriplesNode       ::=       Collection | BlankNodePropertyList
TriplesNode:
	Collection | BlankNodePropertyList;

	//[39]    BlankNodePropertyList     ::=       '[' PropertyListNotEmpty ']'
BlankNodePropertyList:
	'[' PropertyListNotEmpty ']';

	//[40]    Collection    ::=       '(' GraphNode+ ')'
Collection:
	'(' nodes+=GraphNode+ ')';

	//[41]    GraphNode     ::=       VarOrTerm | TriplesNode
GraphNode:
	VarOrTerm | TriplesNode;

	//[42]    VarOrTerm     ::=       Var | GraphTerm
VarOrTerm:
	Var | GraphTerm;

	//[43]    VarOrIRIref       ::=       Var | IRIref
VarOrIRIref:
	Var | IRIref;

	//[44]    Var       ::=       VAR1 | VAR2
Var:
	VAR1 | VAR2;

	//[45]    GraphTerm     ::=       IRIref | RDFLiteral | NumericLiteral | BooleanLiteral | BlankNode | NIL
GraphTerm:
	IRIref | RDFLiteral | {GraphTerm} NumericLiteral | {GraphTerm} BooleanLiteral | BlankNode | {GraphTerm} NIL;

	//[46]    Expression    ::=       ConditionalOrExpression
Expression:
	ConditionalOrExpression;

	//[47]    ConditionalOrExpression       ::=       ConditionalAndExpression ( '||' ConditionalAndExpression )*
ConditionalOrExpression:
	expr+=ConditionalAndExpression ('||' expr+=ConditionalAndExpression)*;

	//[48]    ConditionalAndExpression      ::=       ValueLogical ( '&&' ValueLogical )*
ConditionalAndExpression:
	expr+=ValueLogical ('&&' expr+=ValueLogical)*;

	//[49]    ValueLogical      ::=       RelationalExpression
ValueLogical:
	RelationalExpression;

	//[50]    RelationalExpression      ::=       NumericExpression ( '=' NumericExpression | '!=' NumericExpression | '<' NumericExpression | '>' NumericExpression | '<=' NumericExpression | '>=' NumericExpression )?
RelationalExpression:
	numexpr=NumericExpression ('=' eq=NumericExpression | '!=' neq=NumericExpression | '<' lt=NumericExpression | '>'
	gt=NumericExpression |
	'<=' lte=NumericExpression | '>=' gte=NumericExpression)?;

	//[51]    NumericExpression     ::=       AdditiveExpression
NumericExpression:
	AdditiveExpression;

	//[52]    AdditiveExpression    ::=       MultiplicativeExpression ( '+' MultiplicativeExpression | '-' MultiplicativeExpression | NumericLiteralPositive | NumericLiteralNegative )*
AdditiveExpression:
	expr=MultiplicativeExpression ('+' addExpr+=MultiplicativeExpression | '-' subExpr+=MultiplicativeExpression |
	NumericLiteralPositive | NumericLiteralNegative)*;

	//[53]    MultiplicativeExpression      ::=       UnaryExpression ( '*' UnaryExpression | '/' UnaryExpression )*
MultiplicativeExpression:
	UnaryExpression ('*' timesExpr+=UnaryExpression | '/' divExpr+=UnaryExpression)*;

	//[54]    UnaryExpression       ::=         '!' PrimaryExpression | '+' PrimaryExpression | '-' PrimaryExpression | PrimaryExpression
UnaryExpression:
	'!' PrimaryExpression | '+' PrimaryExpression | '-' PrimaryExpression | PrimaryExpression;

	//[55]    PrimaryExpression     ::=       BrackettedExpression | BuiltInCall | IRIrefOrFunction | RDFLiteral | NumericLiteral | BooleanLiteral | Var
PrimaryExpression:
	BrackettedExpression | BuiltInCall | IRIrefOrFunction | RDFLiteral | num=NumericLiteral | bool=BooleanLiteral | Var;

	//[56]    BrackettedExpression      ::=       '(' Expression ')'
BrackettedExpression:
	'(' Expression ')';

	//[57]    BuiltInCall       ::=         'STR' '(' Expression ')' | 'LANG' '(' Expression ')' | 'LANGMATCHES' '(' Expression ',' Expression ')' | 'DATATYPE' '(' Expression ')' | 'BOUND' '(' Var ')' | 'sameTerm' '(' Expression ',' Expression ')' | 'isIRI' '(' Expression ')' | 'isURI' '(' Expression ')' | 'isBLANK' '(' Expression ')' | 'isLITERAL' '(' Expression ')' | RegexExpression
BuiltInCall:
	'STR' '(' strExpr=Expression ')' | 'LANG' '(' langExpr=Expression ')' | 'LANGMATCHES' '(' langmatchLeft=Expression ','
	langmatchRight=Expression ')' | 'DATATYPE'
	'(' datatypeExpr=Expression ')' | 'BOUND' '(' boundVar=Var ')' | 'sameTerm' '(' sameTermLeft=Expression ','
	sameTermRight=Expression ')' | 'isIRI' '(' isIRIExpr=Expression ')' |
	'isURI' '(' isUriExpr=Expression ')' | 'isBLANK' '(' isBlankExpr=Expression ')' | 'isLITERAL' '('
	isLiteralExpr=Expression ')' | regex=RegexExpression;

	//[58]    RegexExpression       ::=       'REGEX' '(' Expression ',' Expression ( ',' Expression )? ')'
RegexExpression:
	'REGEX' '(' regexLeft=Expression ',' regexMiddle=Expression (',' regexRight=Expression)? ')';

	//[59]    IRIrefOrFunction      ::=       IRIref ArgList?
IRIrefOrFunction:
	iri=IRIref (argList=ArgList)?;

	//[60]    RDFLiteral    ::=       String ( LANGTAG | ( '^^' IRIref ) )?
RDFLiteral:
	string=String (LANGTAG | ('^^' iri=IRIref))?;

	//[61]    NumericLiteral    ::=       NumericLiteralUnsigned | NumericLiteralPositive | NumericLiteralNegative
NumericLiteral:
	NumericLiteralUnsigned | NumericLiteralPositive | NumericLiteralNegative;

	//[62]    NumericLiteralUnsigned    ::=       INTEGER | DECIMAL | DOUBLE
NumericLiteralUnsigned:
	INTEGER | DECIMAL | DOUBLE;

	//[63]    NumericLiteralPositive    ::=       INTEGER_POSITIVE | DECIMAL_POSITIVE | DOUBLE_POSITIVE
NumericLiteralPositive:
	INTEGER_POSITIVE | DECIMAL_POSITIVE | DOUBLE_POSITIVE;

	//[64]    NumericLiteralNegative    ::=       INTEGER_NEGATIVE | DECIMAL_NEGATIVE | DOUBLE_NEGATIVE
NumericLiteralNegative:
	INTEGER_NEGATIVE | DECIMAL_NEGATIVE | DOUBLE_NEGATIVE;

	//[65]    BooleanLiteral    ::=       'true' | 'false'
BooleanLiteral:
	'true' | 'false';

	//[66]    String    ::=       STRING_LITERAL1 | STRING_LITERAL2 | STRING_LITERAL_LONG1 | STRING_LITERAL_LONG2
String:
	STRING_LITERAL1 | STRING_LITERAL2 | STRING_LITERAL_LONG1 | STRING_LITERAL_LONG2;

	//[67]    IRIref    ::=       IRI_REF | PrefixedName
IRIref:
	{IRIref} IRI_REF | PrefixedName;

	//[68]    PrefixedName      ::=       PNAME_LN | PNAME_NS
PrefixedName:
	PNAME_LN | PNAME_NS;

	//[69]    BlankNode     ::=       BLANK_NODE_LABEL | ANON
BlankNode:
	BLANK_NODE_LABEL | label=ANON;

	//[70]    IRI_REF       ::=       '<' ([^<>"{}|^`\]-[#x00-#x20])* '>'
terminal IRI_REF:
	'<' !('<' | '>' | '"' | '{' | '}' | '|' | '^' | '`' | '\\' | '\u0000'..'\u0020')* '>';

	//[71]    PNAME_NS      ::=       PN_PREFIX? ':'
PNAME_NS:
	{PNAME_NS} name=PN_PREFIX? ':';

	//[72]    PNAME_LN      ::=       PNAME_NS PN_LOCAL
PNAME_LN:
	PNAME_NS PN_LOCAL;

	//[73]    BLANK_NODE_LABEL      ::=       '_:' PN_LOCAL
BLANK_NODE_LABEL:
	'_:' label=PN_LOCAL;

	//[74]    VAR1      ::=       '?' VARNAME
VAR1:
	'?' name=VARNAME;

	//[75]    VAR2      ::=       '$' VARNAME
VAR2:
	'$' name=VARNAME;

	//[76]    LANGTAG       ::=       '@' [a-zA-Z]+ ('-' [a-zA-Z0-9]+)*
terminal LANGTAG:
	'@' ('a'..'z' | 'A'..'Z')+ ('-' ('a'..'z' | 'A'..'Z' | '0'..'9')+)*;

	//[77]    INTEGER       ::=       [0-9]+
terminal INTEGER returns ecore::EInt:
	('0'..'9')+;

	//[78]    DECIMAL       ::=       [0-9]+ '.' [0-9]* | '.' [0-9]+
terminal DECIMAL returns ecore::EBigDecimal:
	('0'..'9')+ '.' ('0'..'9')* | '.' ('0'..'9')+;

	//[79]    DOUBLE    ::=       [0-9]+ '.' [0-9]* EXPONENT | '.' ([0-9])+ EXPONENT | ([0-9])+ EXPONENT
terminal DOUBLE returns ecore::EDouble:
	('0'..'9')+ '.' ('0'..'9')* EXPONENT | '.' ('0'..'9')+ EXPONENT | ('0'..'9')+ EXPONENT;

	//[80]    INTEGER_POSITIVE      ::=       '+' INTEGER
terminal INTEGER_POSITIVE returns ecore::EInt:
	'+' INTEGER;

	//[81]    DECIMAL_POSITIVE      ::=       '+' DECIMAL
terminal DECIMAL_POSITIVE returns ecore::EBigDecimal:
	'+' DECIMAL;

	//[82]    DOUBLE_POSITIVE       ::=       '+' DOUBLE
terminal DOUBLE_POSITIVE returns ecore::EDouble:
	'+' DOUBLE;

	//[83]    INTEGER_NEGATIVE      ::=       '-' INTEGER
terminal INTEGER_NEGATIVE returns ecore::EInt:
	'-' INTEGER;

	//[84]    DECIMAL_NEGATIVE      ::=       '-' DECIMAL
terminal DECIMAL_NEGATIVE returns ecore::EBigDecimal:
	'-' DECIMAL;

	//[85]    DOUBLE_NEGATIVE       ::=       '-' DOUBLE
terminal DOUBLE_NEGATIVE returns ecore::EDouble:
	'-' DOUBLE;

	//[86]    EXPONENT      ::=       [eE] [+-]? [0-9]+
terminal EXPONENT:
	('e' | 'E') ('+' | '-')? ('0'..'9')+;

	//[87]    STRING_LITERAL1       ::=       "'" ( ([^#x27#x5C#xA#xD]) | ECHAR )* "'"
terminal STRING_LITERAL1:
	'\'' (!('\u0027' | '\u005C' | '\u000A' | '\u000D') | ECHAR)* '\'';

	//[88]    STRING_LITERAL2       ::=       '"' ( ([^#x22#x5C#xA#xD]) | ECHAR )* '"'
terminal STRING_LITERAL2:
	'\"' (!('\u0022' | '\u005C' | '\u000A' | '\u000D') | ECHAR)* '\"';

	//[89]    STRING_LITERAL_LONG1      ::=       "'''" ( ( "'" | "''" )? ( [^'\] | ECHAR ) )* "'''"
terminal STRING_LITERAL_LONG1:
	"'''" (("'" | "''")? (!("'" | '\\') | ECHAR))* "'''";

	//	//[90]    STRING_LITERAL_LONG2      ::=       '"""' ( ( '"' | '""' )? ( [^"\] | ECHAR ) )* '"""' terminal
terminal STRING_LITERAL_LONG2:
	'"""' (('"' | '""')? (!('"' | '\\') | ECHAR))* '"""';

	//[91]    ECHAR     ::=       '\' [tbnrf\"']
terminal ECHAR:
	'\\' ('t' | 'b' | 'n' | 'r' | 'f' | '\\' | '"' | '\'');

	//[92]    NIL       ::=       '(' WS* ')'
terminal NIL:
	'(' WS* ')';

	//[93]    WS    ::=       #x20 | #x9 | #xD | #xA
terminal WS:
	'\u0020' | '\u0009' | '\u000D' | '\u000A';

	//[94]    ANON      ::=       '[' WS* ']'
terminal ANON:
	'[' WS* ']';

	//[95]    PN_CHARS_BASE ::=		[A-Z] | [a-z] | [#x00C0-#x00D6] | [#x00D8-#x00F6] | [#x00F8-#x02FF] | [#x0370-#x037D] | [#x037F-#x1FFF] | [#x200C-#x200D] | [#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF]
terminal PN_CHARS_BASE:
	'A'..'Z' | 'a'..'z' | '\u00c0'..'\u00d6' | '\u00d8'..'\u00F6' | '\u00F8'..'\u02FF' | '\u0370'..'\u037D' |
	'\u037F'..'\u1FFF' | '\u200C'..'\u200D' | '\u2070'..'\u218F' | '\u2C00'..'\u2FEF' | '\u3001'..'\uD7FF' |
	'\uF900'..'\uFDCF' | '\uFDF0'..'\uFFFD';

	//[96]    PN_CHARS_U    ::=       PN_CHARS_BASE | '_'
terminal PN_CHARS_U:
	PN_CHARS_BASE | '_';

	//[97]    VARNAME       ::=       ( PN_CHARS_U | [0-9] ) ( PN_CHARS_U | [0-9] | #x00B7 | [#x0300-#x036F] | [#x203F-#x2040] )*
terminal VARNAME:
	(PN_CHARS_U | '0'..'9') (PN_CHARS_U | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' | '\u203F'..'\u2040')*;

	//[98]    PN_CHARS      ::=       PN_CHARS_U | '-' | [0-9] | #x00B7 | [#x0300-#x036F] | [#x203F-#x2040]
terminal PN_CHARS:
	PN_CHARS_U | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' | '\u203F'..'\u2040';

	//[99]    PN_PREFIX     ::=       PN_CHARS_BASE ((PN_CHARS|'.')* PN_CHARS)?
terminal PN_PREFIX:
	PN_CHARS_BASE (((PN_CHARS | '.'))* PN_CHARS)?;

	//[100]       PN_LOCAL      ::=       ( PN_CHARS_U | [0-9] ) ((PN_CHARS|'.')* PN_CHARS)?
terminal PN_LOCAL:
	(PN_CHARS_U | '0'..'9') ((PN_CHARS | '.')* PN_CHARS)?;

terminal ANY_OTHER:
	.;


I have attached an image of the runtime eclipse editor that shows the errors:
index.php/fa/21106/0/

The First error says:" extraneous input 'asgh' expecting ':' "
The second one says: " missing RULE_VARNAME at 'a' "
The third error says: " mismatched input 'b' expecting RULE_VARNAME "

I think all of these errors come from some kind of wrong terminal rule definitions but I can't figure out how to make it work.

[edit1] Image wasn't showing properly.

[Updated on: Mon, 09 March 2015 17:45]

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1661639 is a reply to message #1661491] Mon, 09 March 2015 18:26 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
Hi,

please be are: lexing is done before parsing and the lexer is greedy. that means it will eat up as much as it can. and it is context free
you have a very high number of terminals. maybe they are not unambigous. maybe using datatype rules can help

PREFIX abc: <xyz>

will be lexed as

T__31 RULE_WS RULE_VARNAME T__32 RULE_WS RULE_IRI_REF EOF

thus a VARNAME is recognized and not a PN_PREFIX


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: XTEXT does not consume terminals like intended [message #1661783 is a reply to message #1661639] Mon, 09 March 2015 19:52 Go to previous messageGo to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
Hi, thanks for the response Smile

I tried replacing some terminals with production rules but that makes the grammar a non-LL(*) grammar.

(When generating from this grammar i get a lot of ambiguity warnings and the process fails).

I guess that there is no way to express the grammar in xtext since a lot of rules have the same starting terminals (alphanumeric) and the parser cant decide which rule to choose?!
Re: XTEXT does not consume terminals like intended [message #1662617 is a reply to message #1661783] Tue, 10 March 2015 04:15 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
Hi Problem is earlier: Even the Lexer already picks the wrong rules.
another Possibility is to make the grammar less restrictive and add validation to do the restriction


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: XTEXT does not consume terminals like intended [message #1665678 is a reply to message #1662617] Wed, 11 March 2015 10:45 Go to previous messageGo to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
I solved the problem by rewriting the grammar so that each terminal is unique.

Here are the most relevant parts:
	//[95]    PN_CHARS_BASE ::=		[A-Z] | [a-z] | [#x00C0-#x00D6] | [#x00D8-#x00F6] | [#x00F8-#x02FF] | [#x0370-#x037D] | [#x037F-#x1FFF] | [#x200C-#x200D] | [#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF]
terminal PN_CHARS_BASE:
	'A'..'Z' | 'a'..'z' | '\u00c0'..'\u00d6' | '\u00d8'..'\u00F6' | '\u00F8'..'\u02FF' | '\u0370'..'\u037D' |
	'\u037F'..'\u1FFF' | '\u200C'..'\u200D' | '\u2070'..'\u218F' | '\u2C00'..'\u2FEF' | '\u3001'..'\uD7FF' |
	'\uF900'..'\uFDCF' | '\uFDF0'..'\uFFFD';

	//[97]    VARNAME       ::=       ( PN_CHARS_U | [0-9] ) ( PN_CHARS_U | [0-9] | #x00B7 | [#x0300-#x036F] | [#x203F-#x2040] )*
terminal VARIABLE:
	('$' | '?') (PN_CHARS_BASE | '_' | '0'..'9') (PN_CHARS_BASE | '_' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040')*;

terminal PNAME_NS:
	(PN_CHARS_BASE (((PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' | '\u203F'..'\u2040' | '.'))*
	(PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' | '\u203F'..'\u2040'))?)? ':';

	//[72]    PNAME_LN      ::=       PNAME_NS PN_LOCAL
terminal PNAME_LN:
	PNAME_NS ((PN_CHARS_BASE | '_' | '0'..'9') ((PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040' | '.')* (PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040'))?);

	//[73]    BLANK_NODE_LABEL      ::=       '_:' PN_LOCAL
terminal BLANK_NODE_LABEL:
	'_:' ((PN_CHARS_BASE | '_' | '0'..'9') ((PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040' | '.')* (PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040'))?);


But now i run into a different problem:

I would like to reference a defined prefix in the sparql query but i cant use the build in construct to reference it without of changing the PNAME_LN rule.

What i want is a rule like this:
Quote:

//[4] PrefixDecl ::= 'PREFIX' PNAME_NS IRI_REF
PrefixDecl:
'PREFIX' name=PNAME_NS iri=IRI_REF;

PrefixnameLocalPart: name=[PrefixDecl|PNAME_NS] local=PN_LOCAL;

//[72] PNAME_LN ::= PNAME_NS PN_LOCAL
terminal PN_LOCAL:
((PN_CHARS_BASE | '_' | '0'..'9') ((PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
'\u203F'..'\u2040' | '.')* (PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
'\u203F'..'\u2040'))?);


But then the PN_Local rule will again shadow most other terminal rules.

I want this to be able to show all defined prefixes in the content assist of the editor. Is there another way of doing such a thing?
Re: XTEXT does not consume terminals like intended [message #1665753 is a reply to message #1665678] Wed, 11 March 2015 11:26 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
Still the same question: would a data type rule fit as well?

can you please an example model of the model and the reference you want to use?
i do not really get the problem


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

[Updated on: Wed, 11 March 2015 11:43]

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1665805 is a reply to message #1665753] Wed, 11 March 2015 11:57 Go to previous messageGo to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
I don't get the question. You cant use character ranges in datatype rules. As far as I know.

Here is an example model, where the prefix dbpedia_ontology is defined an should be used in the reference in the select query.:
index.php/fa/21131/0/

[Updated on: Wed, 11 March 2015 12:03]

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1665815 is a reply to message #1665805] Wed, 11 March 2015 12:00 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
Hi,

i still do not understand what you actually want to parse
it is ususally done as follows:
have unique terminals and stick them together as datatype rules.
so what is the difference between the new stuff you want to introduce and the existing terminal rules?


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: XTEXT does not consume terminals like intended [message #1665822 is a reply to message #1665815] Wed, 11 March 2015 12:05 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
Hi,

still the question:

xxx:xyz. xxx ist defined which rule? xyz is defined which rule?


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: XTEXT does not consume terminals like intended [message #1665833 is a reply to message #1665815] Wed, 11 March 2015 12:10 Go to previous messageGo to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
The problem is that the terminal rules in SPARQL are not unique and the lexer cant decide which one to use so i have to write terminal that would usually be implemented as datatype rules. This leads to the problem that i cant use a reference for prefixes for example.

(Usually i would do something like this:
Quote:

PrefixDecl: name=ID? ':' iri=IRI_TERMINAL
PrefixNameLocalName: [PrefixDecl] local=LOCALPARTTERMINAL


But the rules for LOCALPARTTERMINAL clash with other terminals like ID, INTEGER and VARNAME
(Because sparql allows for numbers in IDs).

Therefore i had to define the PrefixNameLocalName in a terminal rule so that the ':' is used for uniqueness.

[Updated on: Wed, 11 March 2015 12:15]

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1665835 is a reply to message #1665822] Wed, 11 March 2015 12:14 Go to previous messageGo to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
Quote:

xxx:xyz. xxx ist defined which rule? xyz is defined which rule?


xxx:xyz is defined in the rule PNAME_LN where "xxx:" is defined in PNAME_NS
Re: XTEXT does not consume terminals like intended [message #1665887 is a reply to message #1665835] Wed, 11 March 2015 12:42 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
Hi,

if you have

PrefixDecl: name=ID ':' iri=IRI_TERMINAL

The Reference is

PrefixNameLocalName: ref=[PrefixDecl|ID] ':' local=LOCALPARTTERMINAL

if you have

PrefixDecl: name=DAMN_OTHER_RULE ':' iri=IRI_TERMINAL

The Reference is

PrefixNameLocalName: ref=[PrefixDecl|DAMN_OTHER_RULE] ':' local=LOCALPARTTERMINAL


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: XTEXT does not consume terminals like intended [message #1666112 is a reply to message #1665887] Wed, 11 March 2015 14:49 Go to previous messageGo to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
yeah but that's not the point Very Happy.

And i think i have read somewhere that [XXXRULE] is shorthand for [XXXRULE|ID].

Anyway i solved it with validation and proposal provider.
The grammar looks like follows now:
grammar org.xtext.Sparql hidden(WS)

import 'http://www.eclipse.org/emf/2002/Ecore' as ecore
generate sparql "http://www.xtext.org/Sparql"

//[1]     Query     ::=       Prologue ( SelectQuery | ConstructQuery | DescribeQuery | AskQuery )
Query:
	Prologue (query=SelectQuery | query=ConstructQuery | query=DescribeQuery | query=AskQuery);

	//[2]     Prologue      ::=       BaseDecl? PrefixDecl*
Prologue:
	{Prologue} base=BaseDecl? prefixes+=PrefixDecl*;

	//[3]     BaseDecl      ::=       'BASE' IRI_REF
BaseDecl:
	'BASE' iri=IRI_REF;

	//[4]     PrefixDecl    ::=       'PREFIX' PNAME_NS IRI_REF
PrefixDecl:
	'PREFIX' name=PNAME_NS iri=IRI_REF;

	//[5]     SelectQuery       ::=       'SELECT' ( 'DISTINCT' | 'REDUCED' )? ( Var+ | '*' ) DatasetClause* WhereClause SolutionModifier
SelectQuery:
	'SELECT' ({SelectDistinctQuery} 'DISTINCT' | {SelectReducedQuery} 'REDUCED')? (projectVars+=Var+ | '*')
	dataSets+=DatasetClause* whereClause=WhereClause solutionModifier=SolutionModifier;

	//[6]     ConstructQuery    ::=       'CONSTRUCT' ConstructTemplate DatasetClause* WhereClause SolutionModifier
ConstructQuery:
	'CONSTRUCT' template=ConstructTemplate dataSets+=DatasetClause* whereClause=WhereClause
	solutionmodifier=SolutionModifier;

	//[7]     DescribeQuery     ::=       'DESCRIBE' ( VarOrIRIref+ | '*' ) DatasetClause* WhereClause? SolutionModifier
DescribeQuery:
	'DESCRIBE' (projectvars+=VarOrIRIref+ | '*') dataSets+=DatasetClause* whereClasue=WhereClause?
	solutionModifier=SolutionModifier;

	//[8]     AskQuery      ::=       'ASK' DatasetClause* WhereClause
AskQuery:
	'ASK' (dataSets+=DatasetClause)* whereClause=WhereClause;

	//[9]     DatasetClause     ::=       'FROM' ( DefaultGraphClause | NamedGraphClause )
DatasetClause:
	'FROM' (DefaultGraphClause | NamedGraphClause);

	//[10]    DefaultGraphClause    ::=       SourceSelector
DefaultGraphClause:
	iri=SourceSelector;

	//[11]    NamedGraphClause      ::=       'NAMED' SourceSelector
NamedGraphClause:
	'NAMED' iri=SourceSelector;

	//[12]    SourceSelector    ::=       IRIref
SourceSelector:
	IRIref;

	//[13]    WhereClause       ::=       'WHERE'? GroupGraphPattern
WhereClause:
	{WhereClause} 'WHERE'? groupGraphpattern=GroupGraphPattern;

	//[14]    SolutionModifier      ::=       OrderClause? LimitOffsetClauses?
SolutionModifier:
	{SolutionModifier} (orderClause=OrderClause)? (limitClause=LimitOffsetClauses)?;

	//[15]    LimitOffsetClauses    ::=       ( LimitClause OffsetClause? | OffsetClause LimitClause? )
LimitOffsetClauses:
	(LimitClause OffsetClause? | OffsetClause LimitClause?);

	//[16]    OrderClause       ::=       'ORDER' 'BY' OrderCondition+
OrderClause:
	'ORDER' 'BY' conditions+=OrderCondition+;

	//[17]    OrderCondition    ::=       ( ( 'ASC' | 'DESC' ) BrackettedExpression )| ( Constraint | Var )
OrderCondition:
	'ASC' ascExpr=BrackettedExpression | 'DESC' descExpr=BrackettedExpression | Constraint | Var;

	//[18]    LimitClause       ::=       'LIMIT' INTEGER
LimitClause:
	'LIMIT' INTEGER;

	//[19]    OffsetClause      ::=       'OFFSET' INTEGER
OffsetClause:
	'OFFSET' INTEGER;

	//[20]    GroupGraphPattern     ::=       '{' TriplesBlock? ( ( GraphPatternNotTriples | Filter ) '.'? TriplesBlock? )* '}'
GroupGraphPattern:
	{GroupGraphPattern}
	'{' (triples+=TriplesBlock)? ((graphPattern+=GraphPatternNotTriples | filter+=Filter) '.'? (triples+=TriplesBlock)?)*
	'}';

	//[21]    TriplesBlock      ::=       TriplesSameSubject ( '.' TriplesBlock? )?
TriplesBlock:
	triples+=TriplesSameSubject ('.' (triples+=TriplesBlock)?)?;

	//[22]    GraphPatternNotTriples    ::=       OptionalGraphPattern | GroupOrUnionGraphPattern | GraphGraphPattern
GraphPatternNotTriples:
	optional=OptionalGraphPattern | union=GroupOrUnionGraphPattern | graph=GraphGraphPattern;

	//[23]    OptionalGraphPattern      ::=       'OPTIONAL' GroupGraphPattern
OptionalGraphPattern:
	'OPTIONAL' GroupGraphPattern;

	//[24]    GraphGraphPattern     ::=       'GRAPH' VarOrIRIref GroupGraphPattern
GraphGraphPattern:
	'GRAPH' varOrIri=VarOrIRIref groupgraph=GroupGraphPattern;

	//[25]    GroupOrUnionGraphPattern      ::=       GroupGraphPattern ( 'UNION' GroupGraphPattern )*
GroupOrUnionGraphPattern:
	unionGrupGraph+=GroupGraphPattern ('UNION' unionGrupGraph+=GroupGraphPattern)*;

	//[26]    Filter    ::=       'FILTER' Constraint
Filter:
	'FILTER' Constraint;

	//[27]    Constraint    ::=       BrackettedExpression | BuiltInCall | FunctionCall
Constraint:
	BrackettedExpression | BuiltInCall | FunctionCall;

	//[28]    FunctionCall      ::=       IRIref ArgList
FunctionCall:
	iri=IRIref argList=ArgList;

	//[29]    ArgList       ::=       ( NIL | '(' Expression ( ',' Expression )* ')' )
ArgList:
	({ArgList} NIL | '(' args+=Expression (',' args+=Expression)* ')');

	//[30]    ConstructTemplate     ::=       '{' ConstructTriples? '}'
ConstructTemplate:
	{ConstructTemplate} '{' (constructTriples=ConstructTriples)? '}';

	//[31]    ConstructTriples      ::=       TriplesSameSubject ( '.' ConstructTriples? )?
ConstructTriples:
	TriplesSameSubject ('.' (constructTriples=ConstructTriples)?)?;

	//[32]    TriplesSameSubject    ::=       VarOrTerm PropertyListNotEmpty | TriplesNode PropertyList
TriplesSameSubject:
	subject=VarOrTerm propertyList=PropertyListNotEmpty | subject=TriplesNode propertyList=PropertyList;

	//[33]    PropertyListNotEmpty      ::=       Verb ObjectList ( ';' ( Verb ObjectList )? )*
PropertyListNotEmpty:
	verbs+=Verb objectLists+=ObjectList (';' (verbs+=Verb objectLists+=ObjectList)?)*;

	//[34]    PropertyList      ::=       PropertyListNotEmpty?
PropertyList:
	{PropertyList} (propertyList=PropertyListNotEmpty)?;

	//[35]    ObjectList    ::=       Object ( ',' Object )*
ObjectList:
	objects+=Object (',' objects+=Object)*;

	//[36]    Object    ::=       GraphNode
Object:
	value=GraphNode;

	//[37]    Verb      ::=       VarOrIRIref | 'a'
Verb:
	VarOrIRIref | 'a' {IRIref};

	//[38]    TriplesNode       ::=       Collection | BlankNodePropertyList
TriplesNode:
	Collection | BlankNodePropertyList;

	//[39]    BlankNodePropertyList     ::=       '[' PropertyListNotEmpty ']'
BlankNodePropertyList:
	'[' PropertyListNotEmpty ']';

	//[40]    Collection    ::=       '(' GraphNode+ ')'
Collection:
	'(' nodes+=GraphNode+ ')';

	//[41]    GraphNode     ::=       VarOrTerm | TriplesNode
GraphNode:
	VarOrTerm | TriplesNode;

	//[42]    VarOrTerm     ::=       Var | GraphTerm
VarOrTerm:
	Var | GraphTerm;

	//[43]    VarOrIRIref       ::=       Var | IRIref
VarOrIRIref:
	Var | IRIref;

	//[44]    Var       ::=       VAR1 | VAR2
Var:
	name=VARIABLE;

	//[45]    GraphTerm     ::=       IRIref | RDFLiteral | NumericLiteral | BooleanLiteral | BlankNode | NIL
GraphTerm:
	IRIref | RDFLiteral | {GraphTerm} NumericLiteral | {GraphTerm} BooleanLiteral | BlankNode | {GraphTerm} NIL;

	//[46]    Expression    ::=       ConditionalOrExpression
Expression:
	ConditionalOrExpression;

	//[47]    ConditionalOrExpression       ::=       ConditionalAndExpression ( '||' ConditionalAndExpression )*
ConditionalOrExpression:
	expr+=ConditionalAndExpression ('||' expr+=ConditionalAndExpression)*;

	//[48]    ConditionalAndExpression      ::=       ValueLogical ( '&&' ValueLogical )*
ConditionalAndExpression:
	expr+=ValueLogical ('&&' expr+=ValueLogical)*;

	//[49]    ValueLogical      ::=       RelationalExpression
ValueLogical:
	RelationalExpression;

	//[50]    RelationalExpression      ::=       NumericExpression ( '=' NumericExpression | '!=' NumericExpression | '<' NumericExpression | '>' NumericExpression | '<=' NumericExpression | '>=' NumericExpression )?
RelationalExpression:
	numexpr=NumericExpression ('=' eq=NumericExpression | '!=' neq=NumericExpression | '<' lt=NumericExpression | '>'
	gt=NumericExpression |
	'<=' lte=NumericExpression | '>=' gte=NumericExpression)?;

	//[51]    NumericExpression     ::=       AdditiveExpression
NumericExpression:
	AdditiveExpression;

	//[52]    AdditiveExpression    ::=       MultiplicativeExpression ( '+' MultiplicativeExpression | '-' MultiplicativeExpression | NumericLiteralPositive | NumericLiteralNegative )*
AdditiveExpression:
	expr=MultiplicativeExpression ('+' addExpr+=MultiplicativeExpression | '-' subExpr+=MultiplicativeExpression |
	NumericLiteralPositive | NumericLiteralNegative)*;

	//[53]    MultiplicativeExpression      ::=       UnaryExpression ( '*' UnaryExpression | '/' UnaryExpression )*
MultiplicativeExpression:
	UnaryExpression ('*' timesExpr+=UnaryExpression | '/' divExpr+=UnaryExpression)*;

	//[54]    UnaryExpression       ::=         '!' PrimaryExpression | '+' PrimaryExpression | '-' PrimaryExpression | PrimaryExpression
UnaryExpression:
	'!' PrimaryExpression | '+' PrimaryExpression | '-' PrimaryExpression | PrimaryExpression;

	//[55]    PrimaryExpression     ::=       BrackettedExpression | BuiltInCall | IRIrefOrFunction | RDFLiteral | NumericLiteral | BooleanLiteral | Var
PrimaryExpression:
	BrackettedExpression | BuiltInCall | IRIrefOrFunction | RDFLiteral | num=NumericLiteral | bool=BooleanLiteral | Var;

	//[56]    BrackettedExpression      ::=       '(' Expression ')'
BrackettedExpression:
	'(' Expression ')';

	//[57]    BuiltInCall       ::=         'STR' '(' Expression ')' | 'LANG' '(' Expression ')' | 'LANGMATCHES' '(' Expression ',' Expression ')' | 'DATATYPE' '(' Expression ')' | 'BOUND' '(' Var ')' | 'sameTerm' '(' Expression ',' Expression ')' | 'isIRI' '(' Expression ')' | 'isURI' '(' Expression ')' | 'isBLANK' '(' Expression ')' | 'isLITERAL' '(' Expression ')' | RegexExpression
BuiltInCall:
	'STR' '(' strExpr=Expression ')' | 'LANG' '(' langExpr=Expression ')' | 'LANGMATCHES' '(' langmatchLeft=Expression ','
	langmatchRight=Expression ')' | 'DATATYPE'
	'(' datatypeExpr=Expression ')' | 'BOUND' '(' boundVar=Var ')' | 'sameTerm' '(' sameTermLeft=Expression ','
	sameTermRight=Expression ')' | 'isIRI' '(' isIRIExpr=Expression ')' |
	'isURI' '(' isUriExpr=Expression ')' | 'isBLANK' '(' isBlankExpr=Expression ')' | 'isLITERAL' '('
	isLiteralExpr=Expression ')' | regex=RegexExpression;

	//[58]    RegexExpression       ::=       'REGEX' '(' Expression ',' Expression ( ',' Expression )? ')'
RegexExpression:
	'REGEX' '(' regexLeft=Expression ',' regexMiddle=Expression (',' regexRight=Expression)? ')';

	//[59]    IRIrefOrFunction      ::=       IRIref ArgList?
IRIrefOrFunction:
	iri=IRIref (argList=ArgList)?;

	//[60]    RDFLiteral    ::=       String ( LANGTAG | ( '^^' IRIref ) )?
RDFLiteral:
	string=String (LANGTAG | ('^^' iri=IRIref))?;

	//[61]    NumericLiteral    ::=       NumericLiteralUnsigned | NumericLiteralPositive | NumericLiteralNegative
NumericLiteral:
	NumericLiteralUnsigned | NumericLiteralPositive | NumericLiteralNegative;

	//[62]    NumericLiteralUnsigned    ::=       INTEGER | DECIMAL | DOUBLE
NumericLiteralUnsigned:
	INTEGER | DECIMAL | DOUBLE;

	//[63]    NumericLiteralPositive    ::=       INTEGER_POSITIVE | DECIMAL_POSITIVE | DOUBLE_POSITIVE
NumericLiteralPositive:
	INTEGER_POSITIVE | DECIMAL_POSITIVE | DOUBLE_POSITIVE;

	//[64]    NumericLiteralNegative    ::=       INTEGER_NEGATIVE | DECIMAL_NEGATIVE | DOUBLE_NEGATIVE
NumericLiteralNegative:
	INTEGER_NEGATIVE | DECIMAL_NEGATIVE | DOUBLE_NEGATIVE;

	//[65]    BooleanLiteral    ::=       'true' | 'false'
BooleanLiteral:
	'true' | 'false';

	//[66]    String    ::=       STRING_LITERAL1 | STRING_LITERAL2 | STRING_LITERAL_LONG1 | STRING_LITERAL_LONG2
String:
	STRING_LITERAL1 | STRING_LITERAL2 | STRING_LITERAL_LONG1 | STRING_LITERAL_LONG2;

	//[67]    IRIref    ::=       IRI_REF | PrefixedName
IRIref:
	{IRIref} IRI_REF | PrefixedName;

	//[68]    PrefixedName      ::=       PNAME_LN | PNAME_NS
PrefixedName:
	{PrefixedName} value=PNAME_LN | {PrefixedName} value=PNAME_NS;

	//[69]    BlankNode     ::=       BLANK_NODE_LABEL | ANON
BlankNode:
	label=BLANK_NODE_LABEL | label=ANON;

	//[70]    IRI_REF       ::=       '<' ([^<>"{}|^`\]-[#x00-#x20])* '>'
terminal IRI_REF:
	'<' !('<' | '>' | '"' | '{' | '}' | '|' | '^' | '`' | '\\' | '\u0000'..'\u0020')* '>';

	//[71]    PNAME_NS      ::=       PN_PREFIX? ':'
terminal PNAME_NS:
	(PN_CHARS_BASE (((PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' | '\u203F'..'\u2040' | '.'))*
	(PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' | '\u203F'..'\u2040'))?)? ':';

	//[72]    PNAME_LN      ::=       PNAME_NS PN_LOCAL
terminal PNAME_LN:
	PNAME_NS ((PN_CHARS_BASE | '_' | '0'..'9') ((PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040' | '.')* (PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040'))?);

	//[73]    BLANK_NODE_LABEL      ::=       '_:' PN_LOCAL
terminal BLANK_NODE_LABEL:
	'_:' ((PN_CHARS_BASE | '_' | '0'..'9') ((PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040' | '.')* (PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040'))?);

	//[74]    VAR1      ::=       '?' VARNAME
//[75]    VAR2      ::=       '$' VARNAME
//[76]    LANGTAG       ::=       '@' [a-zA-Z]+ ('-' [a-zA-Z0-9]+)*
terminal LANGTAG:
	'@' ('a'..'z' | 'A'..'Z')+ ('-' ('a'..'z' | 'A'..'Z' | '0'..'9')+)*;

	//[77]    INTEGER       ::=       [0-9]+
terminal INTEGER returns ecore::EInt:
	('0'..'9')+;

	//[78]    DECIMAL       ::=       [0-9]+ '.' [0-9]* | '.' [0-9]+
terminal DECIMAL returns ecore::EBigDecimal:
	('0'..'9')+ '.' ('0'..'9')* | '.' ('0'..'9')+;

	//[79]    DOUBLE    ::=       [0-9]+ '.' [0-9]* EXPONENT | '.' ([0-9])+ EXPONENT | ([0-9])+ EXPONENT
terminal DOUBLE returns ecore::EDouble:
	('0'..'9')+ '.' ('0'..'9')* EXPONENT | '.' ('0'..'9')+ EXPONENT | ('0'..'9')+ EXPONENT;

	//[80]    INTEGER_POSITIVE      ::=       '+' INTEGER
terminal INTEGER_POSITIVE returns ecore::EInt:
	'+' INTEGER;

	//[81]    DECIMAL_POSITIVE      ::=       '+' DECIMAL
terminal DECIMAL_POSITIVE returns ecore::EBigDecimal:
	'+' DECIMAL;

	//[82]    DOUBLE_POSITIVE       ::=       '+' DOUBLE
terminal DOUBLE_POSITIVE returns ecore::EDouble:
	'+' DOUBLE;

	//[83]    INTEGER_NEGATIVE      ::=       '-' INTEGER
terminal INTEGER_NEGATIVE returns ecore::EInt:
	'-' INTEGER;

	//[84]    DECIMAL_NEGATIVE      ::=       '-' DECIMAL
terminal DECIMAL_NEGATIVE returns ecore::EBigDecimal:
	'-' DECIMAL;

	//[85]    DOUBLE_NEGATIVE       ::=       '-' DOUBLE
terminal DOUBLE_NEGATIVE returns ecore::EDouble:
	'-' DOUBLE;

	//[86]    EXPONENT      ::=       [eE] [+-]? [0-9]+
terminal EXPONENT:
	('e' | 'E') ('+' | '-')? ('0'..'9')+;

	//[87]    STRING_LITERAL1       ::=       "'" ( ([^#x27#x5C#xA#xD]) | ECHAR )* "'"
terminal STRING_LITERAL1:
	'\'' (!('\u0027' | '\u005C' | '\u000A' | '\u000D') | ECHAR)* '\'';

	//[88]    STRING_LITERAL2       ::=       '"' ( ([^#x22#x5C#xA#xD]) | ECHAR )* '"'
terminal STRING_LITERAL2:
	'\"' (!('\u0022' | '\u005C' | '\u000A' | '\u000D') | ECHAR)* '\"';

	//[89]    STRING_LITERAL_LONG1      ::=       "'''" ( ( "'" | "''" )? ( [^'\] | ECHAR ) )* "'''"
terminal STRING_LITERAL_LONG1:
	"'''" (("'" | "''")? (!("'" | '\\') | ECHAR))* "'''";

	//	//[90]    STRING_LITERAL_LONG2      ::=       '"""' ( ( '"' | '""' )? ( [^"\] | ECHAR ) )* '"""' terminal
terminal STRING_LITERAL_LONG2:
	'"""' (('"' | '""')? (!('"' | '\\') | ECHAR))* '"""';

	//[91]    ECHAR     ::=       '\' [tbnrf\"']
terminal ECHAR:
	'\\' ('t' | 'b' | 'n' | 'r' | 'f' | '\\' | '"' | '\'');

	//[92]    NIL       ::=       '(' WS* ')'
terminal NIL:
	'(' WS* ')';

	//[93]    WS    ::=       #x20 | #x9 | #xD | #xA
terminal WS:
	'\u0020' | '\u0009' | '\u000D' | '\u000A';

	//[94]    ANON      ::=       '[' WS* ']'
terminal ANON:
	'[' ']';

	//[95]    PN_CHARS_BASE ::=		[A-Z] | [a-z] | [#x00C0-#x00D6] | [#x00D8-#x00F6] | [#x00F8-#x02FF] | [#x0370-#x037D] | [#x037F-#x1FFF] | [#x200C-#x200D] | [#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF]
terminal PN_CHARS_BASE:
	'A'..'Z' | 'a'..'z' | '\u00c0'..'\u00d6' | '\u00d8'..'\u00F6' | '\u00F8'..'\u02FF' | '\u0370'..'\u037D' |
	'\u037F'..'\u1FFF' | '\u200C'..'\u200D' | '\u2070'..'\u218F' | '\u2C00'..'\u2FEF' | '\u3001'..'\uD7FF' |
	'\uF900'..'\uFDCF' | '\uFDF0'..'\uFFFD';

	//[97]    VARNAME       ::=       ( PN_CHARS_U | [0-9] ) ( PN_CHARS_U | [0-9] | #x00B7 | [#x0300-#x036F] | [#x203F-#x2040] )*
terminal VARIABLE:
	('$' | '?') (PN_CHARS_BASE | '_' | '0'..'9') (PN_CHARS_BASE | '_' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040')*;


Here is the validator xtend code:
/*
 * generated by Xtext
 */
package org.xtext.validation

import org.xtext.sparql.PrefixedName
import org.eclipse.xtext.validation.Check
import org.xtext.sparql.Prologue
import org.eclipse.xtext.EcoreUtil2
import org.xtext.sparql.SparqlPackage
import org.xtext.sparql.PrefixDecl

//import org.eclipse.xtext.validation.Check
/**
 * Custom validation rules. 
 *
 * see http://www.eclipse.org/Xtext/documentation.html#validation
 */
class SparqlValidator extends AbstractSparqlValidator {

	//  public static val INVALID_NAME = 'invalidName'
	//
	//	@Check
	//	def checkGreetingStartsWithCapital(Greeting greeting) {
	//		if (!Character.isUpperCase(greeting.name.charAt(0))) {
	//			warning('Name should start with a capital', 
	//					MyDslPackage.Literals.GREETING__NAME,
	//					INVALID_NAME)
	//		}
	//	}
	@Check
	def checkPrefixedNameWasDefined(PrefixedName prefixedName) {
		var prologue = EcoreUtil2.getContainerOfType(prefixedName, Prologue)
		var prefixes = prologue.prefixes;
		var value = prefixedName.value;
		var splitted = value.split(":");
		if (splitted.get(0).equals(":")) {
			value = splitted.get(0);
		} else {
			value = splitted.get(0) + ":";
		}
		val valueVal = value;
		if (prefixes.filter[it.name.equals(valueVal)].empty) {
			error("The Prefix " + valueVal + " is undefined.", SparqlPackage.Literals.PREFIXED_NAME__VALUE);
		}
	}

	@Check
	def checkPrefixIsUnique(PrefixDecl prefixDecl) {
		var prologue = EcoreUtil2.getContainerOfType(prefixDecl, Prologue)
		var prefixes = prologue.prefixes;
		val name = prefixDecl.name;
		if (!prefixes.filter[!it.equals(prefixDecl) && it.name.equals(name)].empty) {
			warning("The Prefix " + name + " is defined more than once.", SparqlPackage.Literals.PREFIX_DECL__NAME);
		}
	}
}

And last but not least the proposal provider:
/*
 * generated by Xtext
 */
package org.xtext.ui.contentassist

import org.eclipse.emf.ecore.EObject
import org.eclipse.xtext.EcoreUtil2
import org.eclipse.xtext.RuleCall
import org.eclipse.xtext.ui.editor.contentassist.ContentAssistContext
import org.eclipse.xtext.ui.editor.contentassist.ICompletionProposalAcceptor
import org.xtext.sparql.Prologue

/**
 * see http://www.eclipse.org/Xtext/documentation.html#contentAssist on how to customize content assistant
 */
class SparqlProposalProvider extends AbstractSparqlProposalProvider {

	override public void complete_PrefixedName(EObject model, RuleCall ruleCall, ContentAssistContext context,
		ICompletionProposalAcceptor acceptor) {
			var prologue=EcoreUtil2.getContainerOfType(model,Prologue)
			var prefixes=prologue.prefixes;
			prefixes.forEach[acceptor.accept(createCompletionProposal(it.name,context))]
	}

}

[Updated on: Wed, 11 March 2015 14:54]

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1676765 is a reply to message #1666112] Sun, 15 March 2015 16:24 Go to previous messageGo to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
I have another questIon regarding this terminals problem:

I would like to write a grammar with rules like this:

Iri: 
    '<' value=ANY_OTHER '>'
;

terminal ANY_OTHER:
    !('\u0000'..'\u0020')*
;

terminal NUM:
    '0'..'9';


So that the value of an IRI can be any character sequence but doesn't overshadow other terminal rules?
Re: XTEXT does not consume terminals like intended [message #1676812 is a reply to message #1676765] Sun, 15 March 2015 16:45 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
hi,

why not simply

Iri:
value=THING
;

terminal THING: '<'->'>';

plus a values converter that strips the <>


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: XTEXT does not consume terminals like intended [message #1676849 is a reply to message #1676812] Sun, 15 March 2015 17:09 Go to previous messageGo to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
So there is no way of defining it with '<' value=SOMETHING '>' so that in the editor i automatically get the '<' and '>' when i want to type an IRI?
Re: XTEXT does not consume terminals like intended [message #1676895 is a reply to message #1676849] Sun, 15 March 2015 17:32 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
hi,

please note: for the editor simply adapt the proposalprovider, there might be a method in the abstract generated supercall you simply have to override. a task that is done in 2 mins


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: XTEXT does not consume terminals like intended [message #1677130 is a reply to message #1676895] Sun, 15 March 2015 19:44 Go to previous messageGo to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
I know that I can use the proposal provider but then the editor does not automatically insert a closing '>' when a opening '<' is typed.

AFAIK the proposal provider is only called when typing strg+space?!
Re: XTEXT does not consume terminals like intended [message #1678143 is a reply to message #1677130] Mon, 16 March 2015 04:58 Go to previous message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
Hi,

this is another feature that is called auto edit (DefaultAutoEditStrategyProvider)


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Previous Topic:create a non xtend generatorFragment (Java)
Next Topic:Auto complete with curly brackets
Goto Forum:
  


Current Time: Fri Apr 16 18:27:35 GMT 2021

Powered by FUDForum. Page generated in 0.03742 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top