Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » XTEXT does not consume terminals like intended
XTEXT does not consume terminals like intended [message #1661491] Mon, 09 March 2015 17:02 Go to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
Hi, Im new to Xtext and I'm trying to get a SPARQL query editor to run.

I have so far "translated" an ebnf grammar for SPARQL into xtext. I'm having trouble with terminal definitions. Eclipse builds the xtext file as intended but when I enter some basic examples into the runtime eclipse editor (started after building the xtext grammar file) i get some confusing errors.

The grammar looks like follows:
grammar org.xtext.Sparql hidden(WS)

import '...' as ecore
generate sparql "..."

//[1]     Query     ::=       Prologue ( SelectQuery | ConstructQuery | DescribeQuery | AskQuery )
Query:
	Prologue (query=SelectQuery | query=ConstructQuery | query=DescribeQuery | query=AskQuery);

	//[2]     Prologue      ::=       BaseDecl? PrefixDecl*
Prologue:
	{Prologue} base=BaseDecl? prefixes+=PrefixDecl*;

	//[3]     BaseDecl      ::=       'BASE' IRI_REF
BaseDecl:
	'BASE' iri=IRI_REF;

	//[4]     PrefixDecl    ::=       'PREFIX' PNAME_NS IRI_REF
PrefixDecl:
	'PREFIX' PNAME_NS iri=IRI_REF;

	//[5]     SelectQuery       ::=       'SELECT' ( 'DISTINCT' | 'REDUCED' )? ( Var+ | '*' ) DatasetClause* WhereClause SolutionModifier
SelectQuery:
	'SELECT' ({SelectDistinctQuery} 'DISTINCT' | {SelectReducedQuery} 'REDUCED')? (projectVars+=Var+ | '*')
	dataSets+=DatasetClause* whereClause=WhereClause solutionModifier=SolutionModifier;

	//[6]     ConstructQuery    ::=       'CONSTRUCT' ConstructTemplate DatasetClause* WhereClause SolutionModifier
ConstructQuery:
	'CONSTRUCT' template=ConstructTemplate dataSets+=DatasetClause* whereClause=WhereClause
	solutionmodifier=SolutionModifier;

	//[7]     DescribeQuery     ::=       'DESCRIBE' ( VarOrIRIref+ | '*' ) DatasetClause* WhereClause? SolutionModifier
DescribeQuery:
	'DESCRIBE' (projectvars+=VarOrIRIref+ | '*') dataSets+=DatasetClause* whereClasue=WhereClause?
	solutionModifier=SolutionModifier;

	//[8]     AskQuery      ::=       'ASK' DatasetClause* WhereClause
AskQuery:
	'ASK' (dataSets+=DatasetClause)* whereClause=WhereClause;

	//[9]     DatasetClause     ::=       'FROM' ( DefaultGraphClause | NamedGraphClause )
DatasetClause:
	'FROM' (DefaultGraphClause | NamedGraphClause);

	//[10]    DefaultGraphClause    ::=       SourceSelector
DefaultGraphClause:
	iri=SourceSelector;

	//[11]    NamedGraphClause      ::=       'NAMED' SourceSelector
NamedGraphClause:
	'NAMED' iri=SourceSelector;

	//[12]    SourceSelector    ::=       IRIref
SourceSelector:
	IRIref;

	//[13]    WhereClause       ::=       'WHERE'? GroupGraphPattern
WhereClause:
	{WhereClause} 'WHERE'? groupGraphpattern=GroupGraphPattern;

	//[14]    SolutionModifier      ::=       OrderClause? LimitOffsetClauses?
SolutionModifier:
	{SolutionModifier} (orderClause=OrderClause)? (limitClause=LimitOffsetClauses)?;

	//[15]    LimitOffsetClauses    ::=       ( LimitClause OffsetClause? | OffsetClause LimitClause? )
LimitOffsetClauses:
	(LimitClause OffsetClause? | OffsetClause LimitClause?);

	//[16]    OrderClause       ::=       'ORDER' 'BY' OrderCondition+
OrderClause:
	'ORDER' 'BY' conditions+=OrderCondition+;

	//[17]    OrderCondition    ::=       ( ( 'ASC' | 'DESC' ) BrackettedExpression )| ( Constraint | Var )
OrderCondition:
	'ASC' ascExpr=BrackettedExpression | 'DESC' descExpr=BrackettedExpression | Constraint | Var;

	//[18]    LimitClause       ::=       'LIMIT' INTEGER
LimitClause:
	'LIMIT' INTEGER;

	//[19]    OffsetClause      ::=       'OFFSET' INTEGER
OffsetClause:
	'OFFSET' INTEGER;

	//[20]    GroupGraphPattern     ::=       '{' TriplesBlock? ( ( GraphPatternNotTriples | Filter ) '.'? TriplesBlock? )* '}'
GroupGraphPattern:
	{GroupGraphPattern}
	'{' (triples+=TriplesBlock)? ((graphPattern+=GraphPatternNotTriples | filter+=Filter) '.'? (triples+=TriplesBlock)?)*
	'}';

	//[21]    TriplesBlock      ::=       TriplesSameSubject ( '.' TriplesBlock? )?
TriplesBlock:
	triples+=TriplesSameSubject ('.' (triples+=TriplesBlock)?)?;

	//[22]    GraphPatternNotTriples    ::=       OptionalGraphPattern | GroupOrUnionGraphPattern | GraphGraphPattern
GraphPatternNotTriples:
	optional=OptionalGraphPattern | union=GroupOrUnionGraphPattern | graph=GraphGraphPattern;

	//[23]    OptionalGraphPattern      ::=       'OPTIONAL' GroupGraphPattern
OptionalGraphPattern:
	'OPTIONAL' GroupGraphPattern;

	//[24]    GraphGraphPattern     ::=       'GRAPH' VarOrIRIref GroupGraphPattern
GraphGraphPattern:
	'GRAPH' varOrIri=VarOrIRIref groupgraph=GroupGraphPattern;

	//[25]    GroupOrUnionGraphPattern      ::=       GroupGraphPattern ( 'UNION' GroupGraphPattern )*
GroupOrUnionGraphPattern:
	unionGrupGraph+=GroupGraphPattern ('UNION' unionGrupGraph+=GroupGraphPattern)*;

	//[26]    Filter    ::=       'FILTER' Constraint
Filter:
	'FILTER' Constraint;

	//[27]    Constraint    ::=       BrackettedExpression | BuiltInCall | FunctionCall
Constraint:
	BrackettedExpression | BuiltInCall | FunctionCall;

	//[28]    FunctionCall      ::=       IRIref ArgList
FunctionCall:
	iri=IRIref argList=ArgList;

	//[29]    ArgList       ::=       ( NIL | '(' Expression ( ',' Expression )* ')' )
ArgList:
	({ArgList} NIL | '(' args+=Expression (',' args+=Expression)* ')');

	//[30]    ConstructTemplate     ::=       '{' ConstructTriples? '}'
ConstructTemplate:
	{ConstructTemplate} '{' (constructTriples=ConstructTriples)? '}';

	//[31]    ConstructTriples      ::=       TriplesSameSubject ( '.' ConstructTriples? )?
ConstructTriples:
	TriplesSameSubject ('.' (constructTriples=ConstructTriples)?)?;

	//[32]    TriplesSameSubject    ::=       VarOrTerm PropertyListNotEmpty | TriplesNode PropertyList
TriplesSameSubject:
	subject=VarOrTerm propertyList=PropertyListNotEmpty | subject=TriplesNode propertyList=PropertyList;

	//[33]    PropertyListNotEmpty      ::=       Verb ObjectList ( ';' ( Verb ObjectList )? )*
PropertyListNotEmpty:
	verbs+=Verb objectLists+=ObjectList (';' (verbs+=Verb objectLists+=ObjectList)?)*;

	//[34]    PropertyList      ::=       PropertyListNotEmpty?
PropertyList:
	{PropertyList} (propertyList=PropertyListNotEmpty)?;

	//[35]    ObjectList    ::=       Object ( ',' Object )*
ObjectList:
	objects+=Object (',' objects+=Object)*;

	//[36]    Object    ::=       GraphNode
Object:
	value=GraphNode;

	//[37]    Verb      ::=       VarOrIRIref | 'a'
Verb:
	VarOrIRIref | 'a' {IRIref};

	//[38]    TriplesNode       ::=       Collection | BlankNodePropertyList
TriplesNode:
	Collection | BlankNodePropertyList;

	//[39]    BlankNodePropertyList     ::=       '[' PropertyListNotEmpty ']'
BlankNodePropertyList:
	'[' PropertyListNotEmpty ']';

	//[40]    Collection    ::=       '(' GraphNode+ ')'
Collection:
	'(' nodes+=GraphNode+ ')';

	//[41]    GraphNode     ::=       VarOrTerm | TriplesNode
GraphNode:
	VarOrTerm | TriplesNode;

	//[42]    VarOrTerm     ::=       Var | GraphTerm
VarOrTerm:
	Var | GraphTerm;

	//[43]    VarOrIRIref       ::=       Var | IRIref
VarOrIRIref:
	Var | IRIref;

	//[44]    Var       ::=       VAR1 | VAR2
Var:
	VAR1 | VAR2;

	//[45]    GraphTerm     ::=       IRIref | RDFLiteral | NumericLiteral | BooleanLiteral | BlankNode | NIL
GraphTerm:
	IRIref | RDFLiteral | {GraphTerm} NumericLiteral | {GraphTerm} BooleanLiteral | BlankNode | {GraphTerm} NIL;

	//[46]    Expression    ::=       ConditionalOrExpression
Expression:
	ConditionalOrExpression;

	//[47]    ConditionalOrExpression       ::=       ConditionalAndExpression ( '||' ConditionalAndExpression )*
ConditionalOrExpression:
	expr+=ConditionalAndExpression ('||' expr+=ConditionalAndExpression)*;

	//[48]    ConditionalAndExpression      ::=       ValueLogical ( '&&' ValueLogical )*
ConditionalAndExpression:
	expr+=ValueLogical ('&&' expr+=ValueLogical)*;

	//[49]    ValueLogical      ::=       RelationalExpression
ValueLogical:
	RelationalExpression;

	//[50]    RelationalExpression      ::=       NumericExpression ( '=' NumericExpression | '!=' NumericExpression | '<' NumericExpression | '>' NumericExpression | '<=' NumericExpression | '>=' NumericExpression )?
RelationalExpression:
	numexpr=NumericExpression ('=' eq=NumericExpression | '!=' neq=NumericExpression | '<' lt=NumericExpression | '>'
	gt=NumericExpression |
	'<=' lte=NumericExpression | '>=' gte=NumericExpression)?;

	//[51]    NumericExpression     ::=       AdditiveExpression
NumericExpression:
	AdditiveExpression;

	//[52]    AdditiveExpression    ::=       MultiplicativeExpression ( '+' MultiplicativeExpression | '-' MultiplicativeExpression | NumericLiteralPositive | NumericLiteralNegative )*
AdditiveExpression:
	expr=MultiplicativeExpression ('+' addExpr+=MultiplicativeExpression | '-' subExpr+=MultiplicativeExpression |
	NumericLiteralPositive | NumericLiteralNegative)*;

	//[53]    MultiplicativeExpression      ::=       UnaryExpression ( '*' UnaryExpression | '/' UnaryExpression )*
MultiplicativeExpression:
	UnaryExpression ('*' timesExpr+=UnaryExpression | '/' divExpr+=UnaryExpression)*;

	//[54]    UnaryExpression       ::=         '!' PrimaryExpression | '+' PrimaryExpression | '-' PrimaryExpression | PrimaryExpression
UnaryExpression:
	'!' PrimaryExpression | '+' PrimaryExpression | '-' PrimaryExpression | PrimaryExpression;

	//[55]    PrimaryExpression     ::=       BrackettedExpression | BuiltInCall | IRIrefOrFunction | RDFLiteral | NumericLiteral | BooleanLiteral | Var
PrimaryExpression:
	BrackettedExpression | BuiltInCall | IRIrefOrFunction | RDFLiteral | num=NumericLiteral | bool=BooleanLiteral | Var;

	//[56]    BrackettedExpression      ::=       '(' Expression ')'
BrackettedExpression:
	'(' Expression ')';

	//[57]    BuiltInCall       ::=         'STR' '(' Expression ')' | 'LANG' '(' Expression ')' | 'LANGMATCHES' '(' Expression ',' Expression ')' | 'DATATYPE' '(' Expression ')' | 'BOUND' '(' Var ')' | 'sameTerm' '(' Expression ',' Expression ')' | 'isIRI' '(' Expression ')' | 'isURI' '(' Expression ')' | 'isBLANK' '(' Expression ')' | 'isLITERAL' '(' Expression ')' | RegexExpression
BuiltInCall:
	'STR' '(' strExpr=Expression ')' | 'LANG' '(' langExpr=Expression ')' | 'LANGMATCHES' '(' langmatchLeft=Expression ','
	langmatchRight=Expression ')' | 'DATATYPE'
	'(' datatypeExpr=Expression ')' | 'BOUND' '(' boundVar=Var ')' | 'sameTerm' '(' sameTermLeft=Expression ','
	sameTermRight=Expression ')' | 'isIRI' '(' isIRIExpr=Expression ')' |
	'isURI' '(' isUriExpr=Expression ')' | 'isBLANK' '(' isBlankExpr=Expression ')' | 'isLITERAL' '('
	isLiteralExpr=Expression ')' | regex=RegexExpression;

	//[58]    RegexExpression       ::=       'REGEX' '(' Expression ',' Expression ( ',' Expression )? ')'
RegexExpression:
	'REGEX' '(' regexLeft=Expression ',' regexMiddle=Expression (',' regexRight=Expression)? ')';

	//[59]    IRIrefOrFunction      ::=       IRIref ArgList?
IRIrefOrFunction:
	iri=IRIref (argList=ArgList)?;

	//[60]    RDFLiteral    ::=       String ( LANGTAG | ( '^^' IRIref ) )?
RDFLiteral:
	string=String (LANGTAG | ('^^' iri=IRIref))?;

	//[61]    NumericLiteral    ::=       NumericLiteralUnsigned | NumericLiteralPositive | NumericLiteralNegative
NumericLiteral:
	NumericLiteralUnsigned | NumericLiteralPositive | NumericLiteralNegative;

	//[62]    NumericLiteralUnsigned    ::=       INTEGER | DECIMAL | DOUBLE
NumericLiteralUnsigned:
	INTEGER | DECIMAL | DOUBLE;

	//[63]    NumericLiteralPositive    ::=       INTEGER_POSITIVE | DECIMAL_POSITIVE | DOUBLE_POSITIVE
NumericLiteralPositive:
	INTEGER_POSITIVE | DECIMAL_POSITIVE | DOUBLE_POSITIVE;

	//[64]    NumericLiteralNegative    ::=       INTEGER_NEGATIVE | DECIMAL_NEGATIVE | DOUBLE_NEGATIVE
NumericLiteralNegative:
	INTEGER_NEGATIVE | DECIMAL_NEGATIVE | DOUBLE_NEGATIVE;

	//[65]    BooleanLiteral    ::=       'true' | 'false'
BooleanLiteral:
	'true' | 'false';

	//[66]    String    ::=       STRING_LITERAL1 | STRING_LITERAL2 | STRING_LITERAL_LONG1 | STRING_LITERAL_LONG2
String:
	STRING_LITERAL1 | STRING_LITERAL2 | STRING_LITERAL_LONG1 | STRING_LITERAL_LONG2;

	//[67]    IRIref    ::=       IRI_REF | PrefixedName
IRIref:
	{IRIref} IRI_REF | PrefixedName;

	//[68]    PrefixedName      ::=       PNAME_LN | PNAME_NS
PrefixedName:
	PNAME_LN | PNAME_NS;

	//[69]    BlankNode     ::=       BLANK_NODE_LABEL | ANON
BlankNode:
	BLANK_NODE_LABEL | label=ANON;

	//[70]    IRI_REF       ::=       '<' ([^<>"{}|^`\]-[#x00-#x20])* '>'
terminal IRI_REF:
	'<' !('<' | '>' | '"' | '{' | '}' | '|' | '^' | '`' | '\\' | '\u0000'..'\u0020')* '>';

	//[71]    PNAME_NS      ::=       PN_PREFIX? ':'
PNAME_NS:
	{PNAME_NS} name=PN_PREFIX? ':';

	//[72]    PNAME_LN      ::=       PNAME_NS PN_LOCAL
PNAME_LN:
	PNAME_NS PN_LOCAL;

	//[73]    BLANK_NODE_LABEL      ::=       '_:' PN_LOCAL
BLANK_NODE_LABEL:
	'_:' label=PN_LOCAL;

	//[74]    VAR1      ::=       '?' VARNAME
VAR1:
	'?' name=VARNAME;

	//[75]    VAR2      ::=       '$' VARNAME
VAR2:
	'$' name=VARNAME;

	//[76]    LANGTAG       ::=       '@' [a-zA-Z]+ ('-' [a-zA-Z0-9]+)*
terminal LANGTAG:
	'@' ('a'..'z' | 'A'..'Z')+ ('-' ('a'..'z' | 'A'..'Z' | '0'..'9')+)*;

	//[77]    INTEGER       ::=       [0-9]+
terminal INTEGER returns ecore::EInt:
	('0'..'9')+;

	//[78]    DECIMAL       ::=       [0-9]+ '.' [0-9]* | '.' [0-9]+
terminal DECIMAL returns ecore::EBigDecimal:
	('0'..'9')+ '.' ('0'..'9')* | '.' ('0'..'9')+;

	//[79]    DOUBLE    ::=       [0-9]+ '.' [0-9]* EXPONENT | '.' ([0-9])+ EXPONENT | ([0-9])+ EXPONENT
terminal DOUBLE returns ecore::EDouble:
	('0'..'9')+ '.' ('0'..'9')* EXPONENT | '.' ('0'..'9')+ EXPONENT | ('0'..'9')+ EXPONENT;

	//[80]    INTEGER_POSITIVE      ::=       '+' INTEGER
terminal INTEGER_POSITIVE returns ecore::EInt:
	'+' INTEGER;

	//[81]    DECIMAL_POSITIVE      ::=       '+' DECIMAL
terminal DECIMAL_POSITIVE returns ecore::EBigDecimal:
	'+' DECIMAL;

	//[82]    DOUBLE_POSITIVE       ::=       '+' DOUBLE
terminal DOUBLE_POSITIVE returns ecore::EDouble:
	'+' DOUBLE;

	//[83]    INTEGER_NEGATIVE      ::=       '-' INTEGER
terminal INTEGER_NEGATIVE returns ecore::EInt:
	'-' INTEGER;

	//[84]    DECIMAL_NEGATIVE      ::=       '-' DECIMAL
terminal DECIMAL_NEGATIVE returns ecore::EBigDecimal:
	'-' DECIMAL;

	//[85]    DOUBLE_NEGATIVE       ::=       '-' DOUBLE
terminal DOUBLE_NEGATIVE returns ecore::EDouble:
	'-' DOUBLE;

	//[86]    EXPONENT      ::=       [eE] [+-]? [0-9]+
terminal EXPONENT:
	('e' | 'E') ('+' | '-')? ('0'..'9')+;

	//[87]    STRING_LITERAL1       ::=       "'" ( ([^#x27#x5C#xA#xD]) | ECHAR )* "'"
terminal STRING_LITERAL1:
	'\'' (!('\u0027' | '\u005C' | '\u000A' | '\u000D') | ECHAR)* '\'';

	//[88]    STRING_LITERAL2       ::=       '"' ( ([^#x22#x5C#xA#xD]) | ECHAR )* '"'
terminal STRING_LITERAL2:
	'\"' (!('\u0022' | '\u005C' | '\u000A' | '\u000D') | ECHAR)* '\"';

	//[89]    STRING_LITERAL_LONG1      ::=       "'''" ( ( "'" | "''" )? ( [^'\] | ECHAR ) )* "'''"
terminal STRING_LITERAL_LONG1:
	"'''" (("'" | "''")? (!("'" | '\\') | ECHAR))* "'''";

	//	//[90]    STRING_LITERAL_LONG2      ::=       '"""' ( ( '"' | '""' )? ( [^"\] | ECHAR ) )* '"""' terminal
terminal STRING_LITERAL_LONG2:
	'"""' (('"' | '""')? (!('"' | '\\') | ECHAR))* '"""';

	//[91]    ECHAR     ::=       '\' [tbnrf\"']
terminal ECHAR:
	'\\' ('t' | 'b' | 'n' | 'r' | 'f' | '\\' | '"' | '\'');

	//[92]    NIL       ::=       '(' WS* ')'
terminal NIL:
	'(' WS* ')';

	//[93]    WS    ::=       #x20 | #x9 | #xD | #xA
terminal WS:
	'\u0020' | '\u0009' | '\u000D' | '\u000A';

	//[94]    ANON      ::=       '[' WS* ']'
terminal ANON:
	'[' WS* ']';

	//[95]    PN_CHARS_BASE ::=		[A-Z] | [a-z] | [#x00C0-#x00D6] | [#x00D8-#x00F6] | [#x00F8-#x02FF] | [#x0370-#x037D] | [#x037F-#x1FFF] | [#x200C-#x200D] | [#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF]
terminal PN_CHARS_BASE:
	'A'..'Z' | 'a'..'z' | '\u00c0'..'\u00d6' | '\u00d8'..'\u00F6' | '\u00F8'..'\u02FF' | '\u0370'..'\u037D' |
	'\u037F'..'\u1FFF' | '\u200C'..'\u200D' | '\u2070'..'\u218F' | '\u2C00'..'\u2FEF' | '\u3001'..'\uD7FF' |
	'\uF900'..'\uFDCF' | '\uFDF0'..'\uFFFD';

	//[96]    PN_CHARS_U    ::=       PN_CHARS_BASE | '_'
terminal PN_CHARS_U:
	PN_CHARS_BASE | '_';

	//[97]    VARNAME       ::=       ( PN_CHARS_U | [0-9] ) ( PN_CHARS_U | [0-9] | #x00B7 | [#x0300-#x036F] | [#x203F-#x2040] )*
terminal VARNAME:
	(PN_CHARS_U | '0'..'9') (PN_CHARS_U | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' | '\u203F'..'\u2040')*;

	//[98]    PN_CHARS      ::=       PN_CHARS_U | '-' | [0-9] | #x00B7 | [#x0300-#x036F] | [#x203F-#x2040]
terminal PN_CHARS:
	PN_CHARS_U | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' | '\u203F'..'\u2040';

	//[99]    PN_PREFIX     ::=       PN_CHARS_BASE ((PN_CHARS|'.')* PN_CHARS)?
terminal PN_PREFIX:
	PN_CHARS_BASE (((PN_CHARS | '.'))* PN_CHARS)?;

	//[100]       PN_LOCAL      ::=       ( PN_CHARS_U | [0-9] ) ((PN_CHARS|'.')* PN_CHARS)?
terminal PN_LOCAL:
	(PN_CHARS_U | '0'..'9') ((PN_CHARS | '.')* PN_CHARS)?;

terminal ANY_OTHER:
	.;


I have attached an image of the runtime eclipse editor that shows the errors:
index.php/fa/21106/0/

The First error says:" extraneous input 'asgh' expecting ':' "
The second one says: " missing RULE_VARNAME at 'a' "
The third error says: " mismatched input 'b' expecting RULE_VARNAME "

I think all of these errors come from some kind of wrong terminal rule definitions but I can't figure out how to make it work.

[edit1] Image wasn't showing properly.

[Updated on: Mon, 09 March 2015 17:45]

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1661639 is a reply to message #1661491] Mon, 09 March 2015 18:26 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14665
Registered: July 2009
Senior Member
Hi,

please be are: lexing is done before parsing and the lexer is greedy. that means it will eat up as much as it can. and it is context free
you have a very high number of terminals. maybe they are not unambigous. maybe using datatype rules can help

PREFIX abc: <xyz>

will be lexed as

T__31 RULE_WS RULE_VARNAME T__32 RULE_WS RULE_IRI_REF EOF

thus a VARNAME is recognized and not a PN_PREFIX

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1661783 is a reply to message #1661639] Mon, 09 March 2015 19:52 Go to previous messageGo to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
Hi, thanks for the response Smile

I tried replacing some terminals with production rules but that makes the grammar a non-LL(*) grammar.

(When generating from this grammar i get a lot of ambiguity warnings and the process fails).

I guess that there is no way to express the grammar in xtext since a lot of rules have the same starting terminals (alphanumeric) and the parser cant decide which rule to choose?!

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1662617 is a reply to message #1661783] Tue, 10 March 2015 04:15 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14665
Registered: July 2009
Senior Member
Hi Problem is earlier: Even the Lexer already picks the wrong rules.
another Possibility is to make the grammar less restrictive and add validation to do the restriction

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1665678 is a reply to message #1662617] Wed, 11 March 2015 10:45 Go to previous messageGo to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
I solved the problem by rewriting the grammar so that each terminal is unique.

Here are the most relevant parts:
	//[95]    PN_CHARS_BASE ::=		[A-Z] | [a-z] | [#x00C0-#x00D6] | [#x00D8-#x00F6] | [#x00F8-#x02FF] | [#x0370-#x037D] | [#x037F-#x1FFF] | [#x200C-#x200D] | [#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF]
terminal PN_CHARS_BASE:
	'A'..'Z' | 'a'..'z' | '\u00c0'..'\u00d6' | '\u00d8'..'\u00F6' | '\u00F8'..'\u02FF' | '\u0370'..'\u037D' |
	'\u037F'..'\u1FFF' | '\u200C'..'\u200D' | '\u2070'..'\u218F' | '\u2C00'..'\u2FEF' | '\u3001'..'\uD7FF' |
	'\uF900'..'\uFDCF' | '\uFDF0'..'\uFFFD';

	//[97]    VARNAME       ::=       ( PN_CHARS_U | [0-9] ) ( PN_CHARS_U | [0-9] | #x00B7 | [#x0300-#x036F] | [#x203F-#x2040] )*
terminal VARIABLE:
	('$' | '?') (PN_CHARS_BASE | '_' | '0'..'9') (PN_CHARS_BASE | '_' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040')*;

terminal PNAME_NS:
	(PN_CHARS_BASE (((PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' | '\u203F'..'\u2040' | '.'))*
	(PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' | '\u203F'..'\u2040'))?)? ':';

	//[72]    PNAME_LN      ::=       PNAME_NS PN_LOCAL
terminal PNAME_LN:
	PNAME_NS ((PN_CHARS_BASE | '_' | '0'..'9') ((PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040' | '.')* (PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040'))?);

	//[73]    BLANK_NODE_LABEL      ::=       '_:' PN_LOCAL
terminal BLANK_NODE_LABEL:
	'_:' ((PN_CHARS_BASE | '_' | '0'..'9') ((PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040' | '.')* (PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040'))?);


But now i run into a different problem:

I would like to reference a defined prefix in the sparql query but i cant use the build in construct to reference it without of changing the PNAME_LN rule.

What i want is a rule like this:
Quote:

//[4] PrefixDecl ::= 'PREFIX' PNAME_NS IRI_REF
PrefixDecl:
'PREFIX' name=PNAME_NS iri=IRI_REF;

PrefixnameLocalPart: name=[PrefixDecl|PNAME_NS] local=PN_LOCAL;

//[72] PNAME_LN ::= PNAME_NS PN_LOCAL
terminal PN_LOCAL:
((PN_CHARS_BASE | '_' | '0'..'9') ((PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
'\u203F'..'\u2040' | '.')* (PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
'\u203F'..'\u2040'))?);


But then the PN_Local rule will again shadow most other terminal rules.

I want this to be able to show all defined prefixes in the content assist of the editor. Is there another way of doing such a thing?

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1665753 is a reply to message #1665678] Wed, 11 March 2015 11:26 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14665
Registered: July 2009
Senior Member
Still the same question: would a data type rule fit as well?

can you please an example model of the model and the reference you want to use?
i do not really get the problem

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

[Updated on: Wed, 11 March 2015 11:43]

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1665805 is a reply to message #1665753] Wed, 11 March 2015 11:57 Go to previous messageGo to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
I don't get the question. You cant use character ranges in datatype rules. As far as I know.

Here is an example model, where the prefix dbpedia_ontology is defined an should be used in the reference in the select query.:
index.php/fa/21131/0/

[Updated on: Wed, 11 March 2015 12:03]

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1665815 is a reply to message #1665805] Wed, 11 March 2015 12:00 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14665
Registered: July 2009
Senior Member
Hi,

i still do not understand what you actually want to parse
it is ususally done as follows:
have unique terminals and stick them together as datatype rules.
so what is the difference between the new stuff you want to introduce and the existing terminal rules?

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1665822 is a reply to message #1665815] Wed, 11 March 2015 12:05 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14665
Registered: July 2009
Senior Member
Hi,

still the question:

xxx:xyz. xxx ist defined which rule? xyz is defined which rule?

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1665833 is a reply to message #1665815] Wed, 11 March 2015 12:10 Go to previous messageGo to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
The problem is that the terminal rules in SPARQL are not unique and the lexer cant decide which one to use so i have to write terminal that would usually be implemented as datatype rules. This leads to the problem that i cant use a reference for prefixes for example.

(Usually i would do something like this:
Quote:

PrefixDecl: name=ID? ':' iri=IRI_TERMINAL
PrefixNameLocalName: [PrefixDecl] local=LOCALPARTTERMINAL


But the rules for LOCALPARTTERMINAL clash with other terminals like ID, INTEGER and VARNAME
(Because sparql allows for numbers in IDs).

Therefore i had to define the PrefixNameLocalName in a terminal rule so that the ':' is used for uniqueness.

[Updated on: Wed, 11 March 2015 12:15]

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1665835 is a reply to message #1665822] Wed, 11 March 2015 12:14 Go to previous messageGo to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
Quote:

xxx:xyz. xxx ist defined which rule? xyz is defined which rule?


xxx:xyz is defined in the rule PNAME_LN where "xxx:" is defined in PNAME_NS

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1665887 is a reply to message #1665835] Wed, 11 March 2015 12:42 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14665
Registered: July 2009
Senior Member
Hi,

if you have

PrefixDecl: name=ID ':' iri=IRI_TERMINAL

The Reference is

PrefixNameLocalName: ref=[PrefixDecl|ID] ':' local=LOCALPARTTERMINAL

if you have

PrefixDecl: name=DAMN_OTHER_RULE ':' iri=IRI_TERMINAL

The Reference is

PrefixNameLocalName: ref=[PrefixDecl|DAMN_OTHER_RULE] ':' local=LOCALPARTTERMINAL

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1666112 is a reply to message #1665887] Wed, 11 March 2015 14:49 Go to previous messageGo to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
yeah but that's not the point Very Happy.

And i think i have read somewhere that [XXXRULE] is shorthand for [XXXRULE|ID].

Anyway i solved it with validation and proposal provider.
The grammar looks like follows now:
grammar org.xtext.Sparql hidden(WS)

import 'http://www.eclipse.org/emf/2002/Ecore' as ecore
generate sparql "http://www.xtext.org/Sparql"

//[1]     Query     ::=       Prologue ( SelectQuery | ConstructQuery | DescribeQuery | AskQuery )
Query:
	Prologue (query=SelectQuery | query=ConstructQuery | query=DescribeQuery | query=AskQuery);

	//[2]     Prologue      ::=       BaseDecl? PrefixDecl*
Prologue:
	{Prologue} base=BaseDecl? prefixes+=PrefixDecl*;

	//[3]     BaseDecl      ::=       'BASE' IRI_REF
BaseDecl:
	'BASE' iri=IRI_REF;

	//[4]     PrefixDecl    ::=       'PREFIX' PNAME_NS IRI_REF
PrefixDecl:
	'PREFIX' name=PNAME_NS iri=IRI_REF;

	//[5]     SelectQuery       ::=       'SELECT' ( 'DISTINCT' | 'REDUCED' )? ( Var+ | '*' ) DatasetClause* WhereClause SolutionModifier
SelectQuery:
	'SELECT' ({SelectDistinctQuery} 'DISTINCT' | {SelectReducedQuery} 'REDUCED')? (projectVars+=Var+ | '*')
	dataSets+=DatasetClause* whereClause=WhereClause solutionModifier=SolutionModifier;

	//[6]     ConstructQuery    ::=       'CONSTRUCT' ConstructTemplate DatasetClause* WhereClause SolutionModifier
ConstructQuery:
	'CONSTRUCT' template=ConstructTemplate dataSets+=DatasetClause* whereClause=WhereClause
	solutionmodifier=SolutionModifier;

	//[7]     DescribeQuery     ::=       'DESCRIBE' ( VarOrIRIref+ | '*' ) DatasetClause* WhereClause? SolutionModifier
DescribeQuery:
	'DESCRIBE' (projectvars+=VarOrIRIref+ | '*') dataSets+=DatasetClause* whereClasue=WhereClause?
	solutionModifier=SolutionModifier;

	//[8]     AskQuery      ::=       'ASK' DatasetClause* WhereClause
AskQuery:
	'ASK' (dataSets+=DatasetClause)* whereClause=WhereClause;

	//[9]     DatasetClause     ::=       'FROM' ( DefaultGraphClause | NamedGraphClause )
DatasetClause:
	'FROM' (DefaultGraphClause | NamedGraphClause);

	//[10]    DefaultGraphClause    ::=       SourceSelector
DefaultGraphClause:
	iri=SourceSelector;

	//[11]    NamedGraphClause      ::=       'NAMED' SourceSelector
NamedGraphClause:
	'NAMED' iri=SourceSelector;

	//[12]    SourceSelector    ::=       IRIref
SourceSelector:
	IRIref;

	//[13]    WhereClause       ::=       'WHERE'? GroupGraphPattern
WhereClause:
	{WhereClause} 'WHERE'? groupGraphpattern=GroupGraphPattern;

	//[14]    SolutionModifier      ::=       OrderClause? LimitOffsetClauses?
SolutionModifier:
	{SolutionModifier} (orderClause=OrderClause)? (limitClause=LimitOffsetClauses)?;

	//[15]    LimitOffsetClauses    ::=       ( LimitClause OffsetClause? | OffsetClause LimitClause? )
LimitOffsetClauses:
	(LimitClause OffsetClause? | OffsetClause LimitClause?);

	//[16]    OrderClause       ::=       'ORDER' 'BY' OrderCondition+
OrderClause:
	'ORDER' 'BY' conditions+=OrderCondition+;

	//[17]    OrderCondition    ::=       ( ( 'ASC' | 'DESC' ) BrackettedExpression )| ( Constraint | Var )
OrderCondition:
	'ASC' ascExpr=BrackettedExpression | 'DESC' descExpr=BrackettedExpression | Constraint | Var;

	//[18]    LimitClause       ::=       'LIMIT' INTEGER
LimitClause:
	'LIMIT' INTEGER;

	//[19]    OffsetClause      ::=       'OFFSET' INTEGER
OffsetClause:
	'OFFSET' INTEGER;

	//[20]    GroupGraphPattern     ::=       '{' TriplesBlock? ( ( GraphPatternNotTriples | Filter ) '.'? TriplesBlock? )* '}'
GroupGraphPattern:
	{GroupGraphPattern}
	'{' (triples+=TriplesBlock)? ((graphPattern+=GraphPatternNotTriples | filter+=Filter) '.'? (triples+=TriplesBlock)?)*
	'}';

	//[21]    TriplesBlock      ::=       TriplesSameSubject ( '.' TriplesBlock? )?
TriplesBlock:
	triples+=TriplesSameSubject ('.' (triples+=TriplesBlock)?)?;

	//[22]    GraphPatternNotTriples    ::=       OptionalGraphPattern | GroupOrUnionGraphPattern | GraphGraphPattern
GraphPatternNotTriples:
	optional=OptionalGraphPattern | union=GroupOrUnionGraphPattern | graph=GraphGraphPattern;

	//[23]    OptionalGraphPattern      ::=       'OPTIONAL' GroupGraphPattern
OptionalGraphPattern:
	'OPTIONAL' GroupGraphPattern;

	//[24]    GraphGraphPattern     ::=       'GRAPH' VarOrIRIref GroupGraphPattern
GraphGraphPattern:
	'GRAPH' varOrIri=VarOrIRIref groupgraph=GroupGraphPattern;

	//[25]    GroupOrUnionGraphPattern      ::=       GroupGraphPattern ( 'UNION' GroupGraphPattern )*
GroupOrUnionGraphPattern:
	unionGrupGraph+=GroupGraphPattern ('UNION' unionGrupGraph+=GroupGraphPattern)*;

	//[26]    Filter    ::=       'FILTER' Constraint
Filter:
	'FILTER' Constraint;

	//[27]    Constraint    ::=       BrackettedExpression | BuiltInCall | FunctionCall
Constraint:
	BrackettedExpression | BuiltInCall | FunctionCall;

	//[28]    FunctionCall      ::=       IRIref ArgList
FunctionCall:
	iri=IRIref argList=ArgList;

	//[29]    ArgList       ::=       ( NIL | '(' Expression ( ',' Expression )* ')' )
ArgList:
	({ArgList} NIL | '(' args+=Expression (',' args+=Expression)* ')');

	//[30]    ConstructTemplate     ::=       '{' ConstructTriples? '}'
ConstructTemplate:
	{ConstructTemplate} '{' (constructTriples=ConstructTriples)? '}';

	//[31]    ConstructTriples      ::=       TriplesSameSubject ( '.' ConstructTriples? )?
ConstructTriples:
	TriplesSameSubject ('.' (constructTriples=ConstructTriples)?)?;

	//[32]    TriplesSameSubject    ::=       VarOrTerm PropertyListNotEmpty | TriplesNode PropertyList
TriplesSameSubject:
	subject=VarOrTerm propertyList=PropertyListNotEmpty | subject=TriplesNode propertyList=PropertyList;

	//[33]    PropertyListNotEmpty      ::=       Verb ObjectList ( ';' ( Verb ObjectList )? )*
PropertyListNotEmpty:
	verbs+=Verb objectLists+=ObjectList (';' (verbs+=Verb objectLists+=ObjectList)?)*;

	//[34]    PropertyList      ::=       PropertyListNotEmpty?
PropertyList:
	{PropertyList} (propertyList=PropertyListNotEmpty)?;

	//[35]    ObjectList    ::=       Object ( ',' Object )*
ObjectList:
	objects+=Object (',' objects+=Object)*;

	//[36]    Object    ::=       GraphNode
Object:
	value=GraphNode;

	//[37]    Verb      ::=       VarOrIRIref | 'a'
Verb:
	VarOrIRIref | 'a' {IRIref};

	//[38]    TriplesNode       ::=       Collection | BlankNodePropertyList
TriplesNode:
	Collection | BlankNodePropertyList;

	//[39]    BlankNodePropertyList     ::=       '[' PropertyListNotEmpty ']'
BlankNodePropertyList:
	'[' PropertyListNotEmpty ']';

	//[40]    Collection    ::=       '(' GraphNode+ ')'
Collection:
	'(' nodes+=GraphNode+ ')';

	//[41]    GraphNode     ::=       VarOrTerm | TriplesNode
GraphNode:
	VarOrTerm | TriplesNode;

	//[42]    VarOrTerm     ::=       Var | GraphTerm
VarOrTerm:
	Var | GraphTerm;

	//[43]    VarOrIRIref       ::=       Var | IRIref
VarOrIRIref:
	Var | IRIref;

	//[44]    Var       ::=       VAR1 | VAR2
Var:
	name=VARIABLE;

	//[45]    GraphTerm     ::=       IRIref | RDFLiteral | NumericLiteral | BooleanLiteral | BlankNode | NIL
GraphTerm:
	IRIref | RDFLiteral | {GraphTerm} NumericLiteral | {GraphTerm} BooleanLiteral | BlankNode | {GraphTerm} NIL;

	//[46]    Expression    ::=       ConditionalOrExpression
Expression:
	ConditionalOrExpression;

	//[47]    ConditionalOrExpression       ::=       ConditionalAndExpression ( '||' ConditionalAndExpression )*
ConditionalOrExpression:
	expr+=ConditionalAndExpression ('||' expr+=ConditionalAndExpression)*;

	//[48]    ConditionalAndExpression      ::=       ValueLogical ( '&&' ValueLogical )*
ConditionalAndExpression:
	expr+=ValueLogical ('&&' expr+=ValueLogical)*;

	//[49]    ValueLogical      ::=       RelationalExpression
ValueLogical:
	RelationalExpression;

	//[50]    RelationalExpression      ::=       NumericExpression ( '=' NumericExpression | '!=' NumericExpression | '<' NumericExpression | '>' NumericExpression | '<=' NumericExpression | '>=' NumericExpression )?
RelationalExpression:
	numexpr=NumericExpression ('=' eq=NumericExpression | '!=' neq=NumericExpression | '<' lt=NumericExpression | '>'
	gt=NumericExpression |
	'<=' lte=NumericExpression | '>=' gte=NumericExpression)?;

	//[51]    NumericExpression     ::=       AdditiveExpression
NumericExpression:
	AdditiveExpression;

	//[52]    AdditiveExpression    ::=       MultiplicativeExpression ( '+' MultiplicativeExpression | '-' MultiplicativeExpression | NumericLiteralPositive | NumericLiteralNegative )*
AdditiveExpression:
	expr=MultiplicativeExpression ('+' addExpr+=MultiplicativeExpression | '-' subExpr+=MultiplicativeExpression |
	NumericLiteralPositive | NumericLiteralNegative)*;

	//[53]    MultiplicativeExpression      ::=       UnaryExpression ( '*' UnaryExpression | '/' UnaryExpression )*
MultiplicativeExpression:
	UnaryExpression ('*' timesExpr+=UnaryExpression | '/' divExpr+=UnaryExpression)*;

	//[54]    UnaryExpression       ::=         '!' PrimaryExpression | '+' PrimaryExpression | '-' PrimaryExpression | PrimaryExpression
UnaryExpression:
	'!' PrimaryExpression | '+' PrimaryExpression | '-' PrimaryExpression | PrimaryExpression;

	//[55]    PrimaryExpression     ::=       BrackettedExpression | BuiltInCall | IRIrefOrFunction | RDFLiteral | NumericLiteral | BooleanLiteral | Var
PrimaryExpression:
	BrackettedExpression | BuiltInCall | IRIrefOrFunction | RDFLiteral | num=NumericLiteral | bool=BooleanLiteral | Var;

	//[56]    BrackettedExpression      ::=       '(' Expression ')'
BrackettedExpression:
	'(' Expression ')';

	//[57]    BuiltInCall       ::=         'STR' '(' Expression ')' | 'LANG' '(' Expression ')' | 'LANGMATCHES' '(' Expression ',' Expression ')' | 'DATATYPE' '(' Expression ')' | 'BOUND' '(' Var ')' | 'sameTerm' '(' Expression ',' Expression ')' | 'isIRI' '(' Expression ')' | 'isURI' '(' Expression ')' | 'isBLANK' '(' Expression ')' | 'isLITERAL' '(' Expression ')' | RegexExpression
BuiltInCall:
	'STR' '(' strExpr=Expression ')' | 'LANG' '(' langExpr=Expression ')' | 'LANGMATCHES' '(' langmatchLeft=Expression ','
	langmatchRight=Expression ')' | 'DATATYPE'
	'(' datatypeExpr=Expression ')' | 'BOUND' '(' boundVar=Var ')' | 'sameTerm' '(' sameTermLeft=Expression ','
	sameTermRight=Expression ')' | 'isIRI' '(' isIRIExpr=Expression ')' |
	'isURI' '(' isUriExpr=Expression ')' | 'isBLANK' '(' isBlankExpr=Expression ')' | 'isLITERAL' '('
	isLiteralExpr=Expression ')' | regex=RegexExpression;

	//[58]    RegexExpression       ::=       'REGEX' '(' Expression ',' Expression ( ',' Expression )? ')'
RegexExpression:
	'REGEX' '(' regexLeft=Expression ',' regexMiddle=Expression (',' regexRight=Expression)? ')';

	//[59]    IRIrefOrFunction      ::=       IRIref ArgList?
IRIrefOrFunction:
	iri=IRIref (argList=ArgList)?;

	//[60]    RDFLiteral    ::=       String ( LANGTAG | ( '^^' IRIref ) )?
RDFLiteral:
	string=String (LANGTAG | ('^^' iri=IRIref))?;

	//[61]    NumericLiteral    ::=       NumericLiteralUnsigned | NumericLiteralPositive | NumericLiteralNegative
NumericLiteral:
	NumericLiteralUnsigned | NumericLiteralPositive | NumericLiteralNegative;

	//[62]    NumericLiteralUnsigned    ::=       INTEGER | DECIMAL | DOUBLE
NumericLiteralUnsigned:
	INTEGER | DECIMAL | DOUBLE;

	//[63]    NumericLiteralPositive    ::=       INTEGER_POSITIVE | DECIMAL_POSITIVE | DOUBLE_POSITIVE
NumericLiteralPositive:
	INTEGER_POSITIVE | DECIMAL_POSITIVE | DOUBLE_POSITIVE;

	//[64]    NumericLiteralNegative    ::=       INTEGER_NEGATIVE | DECIMAL_NEGATIVE | DOUBLE_NEGATIVE
NumericLiteralNegative:
	INTEGER_NEGATIVE | DECIMAL_NEGATIVE | DOUBLE_NEGATIVE;

	//[65]    BooleanLiteral    ::=       'true' | 'false'
BooleanLiteral:
	'true' | 'false';

	//[66]    String    ::=       STRING_LITERAL1 | STRING_LITERAL2 | STRING_LITERAL_LONG1 | STRING_LITERAL_LONG2
String:
	STRING_LITERAL1 | STRING_LITERAL2 | STRING_LITERAL_LONG1 | STRING_LITERAL_LONG2;

	//[67]    IRIref    ::=       IRI_REF | PrefixedName
IRIref:
	{IRIref} IRI_REF | PrefixedName;

	//[68]    PrefixedName      ::=       PNAME_LN | PNAME_NS
PrefixedName:
	{PrefixedName} value=PNAME_LN | {PrefixedName} value=PNAME_NS;

	//[69]    BlankNode     ::=       BLANK_NODE_LABEL | ANON
BlankNode:
	label=BLANK_NODE_LABEL | label=ANON;

	//[70]    IRI_REF       ::=       '<' ([^<>"{}|^`\]-[#x00-#x20])* '>'
terminal IRI_REF:
	'<' !('<' | '>' | '"' | '{' | '}' | '|' | '^' | '`' | '\\' | '\u0000'..'\u0020')* '>';

	//[71]    PNAME_NS      ::=       PN_PREFIX? ':'
terminal PNAME_NS:
	(PN_CHARS_BASE (((PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' | '\u203F'..'\u2040' | '.'))*
	(PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' | '\u203F'..'\u2040'))?)? ':';

	//[72]    PNAME_LN      ::=       PNAME_NS PN_LOCAL
terminal PNAME_LN:
	PNAME_NS ((PN_CHARS_BASE | '_' | '0'..'9') ((PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040' | '.')* (PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040'))?);

	//[73]    BLANK_NODE_LABEL      ::=       '_:' PN_LOCAL
terminal BLANK_NODE_LABEL:
	'_:' ((PN_CHARS_BASE | '_' | '0'..'9') ((PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040' | '.')* (PN_CHARS_BASE | '_' | '-' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040'))?);

	//[74]    VAR1      ::=       '?' VARNAME
//[75]    VAR2      ::=       '$' VARNAME
//[76]    LANGTAG       ::=       '@' [a-zA-Z]+ ('-' [a-zA-Z0-9]+)*
terminal LANGTAG:
	'@' ('a'..'z' | 'A'..'Z')+ ('-' ('a'..'z' | 'A'..'Z' | '0'..'9')+)*;

	//[77]    INTEGER       ::=       [0-9]+
terminal INTEGER returns ecore::EInt:
	('0'..'9')+;

	//[78]    DECIMAL       ::=       [0-9]+ '.' [0-9]* | '.' [0-9]+
terminal DECIMAL returns ecore::EBigDecimal:
	('0'..'9')+ '.' ('0'..'9')* | '.' ('0'..'9')+;

	//[79]    DOUBLE    ::=       [0-9]+ '.' [0-9]* EXPONENT | '.' ([0-9])+ EXPONENT | ([0-9])+ EXPONENT
terminal DOUBLE returns ecore::EDouble:
	('0'..'9')+ '.' ('0'..'9')* EXPONENT | '.' ('0'..'9')+ EXPONENT | ('0'..'9')+ EXPONENT;

	//[80]    INTEGER_POSITIVE      ::=       '+' INTEGER
terminal INTEGER_POSITIVE returns ecore::EInt:
	'+' INTEGER;

	//[81]    DECIMAL_POSITIVE      ::=       '+' DECIMAL
terminal DECIMAL_POSITIVE returns ecore::EBigDecimal:
	'+' DECIMAL;

	//[82]    DOUBLE_POSITIVE       ::=       '+' DOUBLE
terminal DOUBLE_POSITIVE returns ecore::EDouble:
	'+' DOUBLE;

	//[83]    INTEGER_NEGATIVE      ::=       '-' INTEGER
terminal INTEGER_NEGATIVE returns ecore::EInt:
	'-' INTEGER;

	//[84]    DECIMAL_NEGATIVE      ::=       '-' DECIMAL
terminal DECIMAL_NEGATIVE returns ecore::EBigDecimal:
	'-' DECIMAL;

	//[85]    DOUBLE_NEGATIVE       ::=       '-' DOUBLE
terminal DOUBLE_NEGATIVE returns ecore::EDouble:
	'-' DOUBLE;

	//[86]    EXPONENT      ::=       [eE] [+-]? [0-9]+
terminal EXPONENT:
	('e' | 'E') ('+' | '-')? ('0'..'9')+;

	//[87]    STRING_LITERAL1       ::=       "'" ( ([^#x27#x5C#xA#xD]) | ECHAR )* "'"
terminal STRING_LITERAL1:
	'\'' (!('\u0027' | '\u005C' | '\u000A' | '\u000D') | ECHAR)* '\'';

	//[88]    STRING_LITERAL2       ::=       '"' ( ([^#x22#x5C#xA#xD]) | ECHAR )* '"'
terminal STRING_LITERAL2:
	'\"' (!('\u0022' | '\u005C' | '\u000A' | '\u000D') | ECHAR)* '\"';

	//[89]    STRING_LITERAL_LONG1      ::=       "'''" ( ( "'" | "''" )? ( [^'\] | ECHAR ) )* "'''"
terminal STRING_LITERAL_LONG1:
	"'''" (("'" | "''")? (!("'" | '\\') | ECHAR))* "'''";

	//	//[90]    STRING_LITERAL_LONG2      ::=       '"""' ( ( '"' | '""' )? ( [^"\] | ECHAR ) )* '"""' terminal
terminal STRING_LITERAL_LONG2:
	'"""' (('"' | '""')? (!('"' | '\\') | ECHAR))* '"""';

	//[91]    ECHAR     ::=       '\' [tbnrf\"']
terminal ECHAR:
	'\\' ('t' | 'b' | 'n' | 'r' | 'f' | '\\' | '"' | '\'');

	//[92]    NIL       ::=       '(' WS* ')'
terminal NIL:
	'(' WS* ')';

	//[93]    WS    ::=       #x20 | #x9 | #xD | #xA
terminal WS:
	'\u0020' | '\u0009' | '\u000D' | '\u000A';

	//[94]    ANON      ::=       '[' WS* ']'
terminal ANON:
	'[' ']';

	//[95]    PN_CHARS_BASE ::=		[A-Z] | [a-z] | [#x00C0-#x00D6] | [#x00D8-#x00F6] | [#x00F8-#x02FF] | [#x0370-#x037D] | [#x037F-#x1FFF] | [#x200C-#x200D] | [#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF]
terminal PN_CHARS_BASE:
	'A'..'Z' | 'a'..'z' | '\u00c0'..'\u00d6' | '\u00d8'..'\u00F6' | '\u00F8'..'\u02FF' | '\u0370'..'\u037D' |
	'\u037F'..'\u1FFF' | '\u200C'..'\u200D' | '\u2070'..'\u218F' | '\u2C00'..'\u2FEF' | '\u3001'..'\uD7FF' |
	'\uF900'..'\uFDCF' | '\uFDF0'..'\uFFFD';

	//[97]    VARNAME       ::=       ( PN_CHARS_U | [0-9] ) ( PN_CHARS_U | [0-9] | #x00B7 | [#x0300-#x036F] | [#x203F-#x2040] )*
terminal VARIABLE:
	('$' | '?') (PN_CHARS_BASE | '_' | '0'..'9') (PN_CHARS_BASE | '_' | '0'..'9' | '\u00B7' | '\u0300'..'\u036F' |
	'\u203F'..'\u2040')*;


Here is the validator xtend code:
/*
 * generated by Xtext
 */
package org.xtext.validation

import org.xtext.sparql.PrefixedName
import org.eclipse.xtext.validation.Check
import org.xtext.sparql.Prologue
import org.eclipse.xtext.EcoreUtil2
import org.xtext.sparql.SparqlPackage
import org.xtext.sparql.PrefixDecl

//import org.eclipse.xtext.validation.Check
/**
 * Custom validation rules. 
 *
 * see http://www.eclipse.org/Xtext/documentation.html#validation
 */
class SparqlValidator extends AbstractSparqlValidator {

	//  public static val INVALID_NAME = 'invalidName'
	//
	//	@Check
	//	def checkGreetingStartsWithCapital(Greeting greeting) {
	//		if (!Character.isUpperCase(greeting.name.charAt(0))) {
	//			warning('Name should start with a capital', 
	//					MyDslPackage.Literals.GREETING__NAME,
	//					INVALID_NAME)
	//		}
	//	}
	@Check
	def checkPrefixedNameWasDefined(PrefixedName prefixedName) {
		var prologue = EcoreUtil2.getContainerOfType(prefixedName, Prologue)
		var prefixes = prologue.prefixes;
		var value = prefixedName.value;
		var splitted = value.split(":");
		if (splitted.get(0).equals(":")) {
			value = splitted.get(0);
		} else {
			value = splitted.get(0) + ":";
		}
		val valueVal = value;
		if (prefixes.filter[it.name.equals(valueVal)].empty) {
			error("The Prefix " + valueVal + " is undefined.", SparqlPackage.Literals.PREFIXED_NAME__VALUE);
		}
	}

	@Check
	def checkPrefixIsUnique(PrefixDecl prefixDecl) {
		var prologue = EcoreUtil2.getContainerOfType(prefixDecl, Prologue)
		var prefixes = prologue.prefixes;
		val name = prefixDecl.name;
		if (!prefixes.filter[!it.equals(prefixDecl) && it.name.equals(name)].empty) {
			warning("The Prefix " + name + " is defined more than once.", SparqlPackage.Literals.PREFIX_DECL__NAME);
		}
	}
}

And last but not least the proposal provider:
/*
 * generated by Xtext
 */
package org.xtext.ui.contentassist

import org.eclipse.emf.ecore.EObject
import org.eclipse.xtext.EcoreUtil2
import org.eclipse.xtext.RuleCall
import org.eclipse.xtext.ui.editor.contentassist.ContentAssistContext
import org.eclipse.xtext.ui.editor.contentassist.ICompletionProposalAcceptor
import org.xtext.sparql.Prologue

/**
 * see http://www.eclipse.org/Xtext/documentation.html#contentAssist on how to customize content assistant
 */
class SparqlProposalProvider extends AbstractSparqlProposalProvider {

	override public void complete_PrefixedName(EObject model, RuleCall ruleCall, ContentAssistContext context,
		ICompletionProposalAcceptor acceptor) {
			var prologue=EcoreUtil2.getContainerOfType(model,Prologue)
			var prefixes=prologue.prefixes;
			prefixes.forEach[acceptor.accept(createCompletionProposal(it.name,context))]
	}

}

[Updated on: Wed, 11 March 2015 14:54]

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1676765 is a reply to message #1666112] Sun, 15 March 2015 16:24 Go to previous messageGo to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
I have another questIon regarding this terminals problem:

I would like to write a grammar with rules like this:

Iri: 
    '<' value=ANY_OTHER '>'
;

terminal ANY_OTHER:
    !('\u0000'..'\u0020')*
;

terminal NUM:
    '0'..'9';


So that the value of an IRI can be any character sequence but doesn't overshadow other terminal rules?

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1676812 is a reply to message #1676765] Sun, 15 March 2015 16:45 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14665
Registered: July 2009
Senior Member
hi,

why not simply

Iri:
value=THING
;

terminal THING: '<'->'>';

plus a values converter that strips the <>

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1676849 is a reply to message #1676812] Sun, 15 March 2015 17:09 Go to previous messageGo to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
So there is no way of defining it with '<' value=SOMETHING '>' so that in the editor i automatically get the '<' and '>' when i want to type an IRI?

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1676895 is a reply to message #1676849] Sun, 15 March 2015 17:32 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14665
Registered: July 2009
Senior Member
hi,

please note: for the editor simply adapt the proposalprovider, there might be a method in the abstract generated supercall you simply have to override. a task that is done in 2 mins

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1677130 is a reply to message #1676895] Sun, 15 March 2015 19:44 Go to previous messageGo to next message
QW DR is currently offline QW DRFriend
Messages: 11
Registered: March 2015
Junior Member
I know that I can use the proposal provider but then the editor does not automatically insert a closing '>' when a opening '<' is typed.

AFAIK the proposal provider is only called when typing strg+space?!

Report message to a moderator

Re: XTEXT does not consume terminals like intended [message #1678143 is a reply to message #1677130] Mon, 16 March 2015 04:58 Go to previous message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14665
Registered: July 2009
Senior Member
Hi,

this is another feature that is called auto edit (DefaultAutoEditStrategyProvider)

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Previous Topic:create a non xtend generatorFragment (Java)
Next Topic:Auto complete with curly brackets
Goto Forum:
  

[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Wed Apr 24 15:26:12 GMT 2024

Powered by FUDForum. Page generated in 0.04836 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top

Sign up to our Newsletter

A fresh new issue delivered monthly