Eclipse Community Forums: TMF (Xtext) » Content Assist Contexts

Help

Home

Home » Modeling » TMF (Xtext) » Content Assist Contexts

Show: Today's Messages :: Show Polls :: Message Navigator

Content Assist Contexts [message #1696834]

Thu, 28 May 2015 21:41

Luis De Bello

Messages: 95
Registered: January 2015

Member

Hi guys,

I am having some issues with the content assist it works great but there are some scenarios where it does not work because the code is not able to create a "ContentAssistContext" I was debugging the code and the result of "createContentAssistContexts" which is invoke from CompletionProposalComputer returns an empty array.

This work ok:
{
testing: {
inside1: {
message: model.
}
}
}

However if I try something similar it does not work

{
testing: [
inside1: {
message: model.
}
]
}

I know that it depends on the grammar but I wanted to ask you if you know if there are some material/documentation to get a better understanding on how handle the context, maybe if someone has some experience dealing with some similar to check if the logic provide by Xtext is usually enough or I should check if I need to create my own logic to create these contexts.

Thanks in advance.

Regards,
Luis

Report message to a moderator

Re: Content Assist Contexts [message #1696862 is a reply to message #1696834]

Fri, 29 May 2015 07:16

Sven Efftinge

Messages: 1823
Registered: July 2009

Senior Member

In order to help, we would need to see the grammar as well.
Also in the examples you need to point out where the cursor is, when you trigger content assist.

Sven

Report message to a moderator

Re: Content Assist Contexts [message #1696936 is a reply to message #1696862]

Fri, 29 May 2015 13:55

Luis De Bello

Messages: 95
Registered: January 2015

Member

Hi Sven,

Thanks for your answer, I enclose the grammar and the examples with the cursor:

This work ok:
{
testing: {
inside1: {
message: model. @CursorHere@
}
}
}

However if I try something similar it does not work
{
testing: [
inside1: {
message: model. @CursorHere@
}
]
}

Grammar:

grammar org.designer.tooling.testing.DFL hidden(WS, EOL, SL_COMMENT)

import "http://www.eclipse.org/emf/2002/Ecore" as ecore

generate dFL "http://www.designer.org/tooling/testing/DFL"

/*
* Document
*/
Document:
body=Body?;

Body:
header=Header?
content=Content;

/*
* Header
*/
Header:
directives+=Directive+ separator=HeaderSeparator;

HeaderSeparator:
separator="---";

Directive:
VersionDirective | NamespaceDirective | VariableDirective | OutputDirective | InputDirective | DebugDirective |
TypeDirective | FunctionDirective | AstDirective;

VersionDirective:
version=VersionNumber;

VersionNumber hidden(EOL, SL_COMMENT):
"%weave" WS+ INT "." INT;

NamespaceDirective:
"%namespace" name=NamespacePrefix value=NamespaceUri;

VariableDirective:
'%var' name=Identifier "=" constantValue=VariableValue;

VariableValue:
FunctionLiteral | Expression;

OutputDirective:
"%output" type=DataType options=Options?;

TypeDirective:
"%type" name=Identifier '=' type=Type;

AstDirective:
type="%ast";

/*
* Functions
*/
FunctionDirective:
"%function" name=Identifier "(" (parameters+=FunctionParameter ("," parameters+=FunctionParameter)*)? ")"
expression=Expression;

Schema:
{Schema} '{' (elements+=SchemaElement ("," elements+=SchemaElement)*)? '}';

SchemaElement:
name=Identifier ':' value=LiteralValue;

InputDirective:
"%input" name=Identifier type=DataType options=Options?;

DebugDirective:
{DebugDirective} "%debug" options=Options?;

Options:
options+=OptionElement ("," options+=OptionElement)*;

OptionElement:
name=Identifier "=" value=LiteralValue;

/*
* Content
*/
Content:
elements+=Expression;

MultipleKeyValuePairObj:
{MultipleKeyValuePairObj} "{" (elements+=ObjectElement ("," elements+=ObjectElement?)*)? "}";

ObjectElement:
=> ConditionalKeyValuePair | => KeyValuePair | EnclosedExpression;

KeyValuePair:
key=Key ":" (-> value=Expression?);

ConditionalKeyValuePair:
"(" key=Key ":" value=Expression ")" "when" condition=Expression;

/**
* Key
*/
Key:
namespace=DeclaredNamespace? value=KeyExpression attributes=Attributes?;

KeyExpression:
({StringLiteral} value=StringLiteral | {RealLiteral} value=RealLiteral | {IntegerLiteral} value=IntegerLiteral |
{BooleanLiteral} value=BooleanLiteral | {AnyDateLiteral} value=AnyDateLiteral) |
(EnclosedExpression => (
// We only dispatch one specific action for each selector because dispatching different actions consume a lot of time during the generation of EMF model, lexer and parser

// Descendant Selectors
{DescendantSelector.data=current} type=(".." | "..@" | "..^") (=> selector=Selector)? |

// Dot Selectors
{DotSelector.data=current} (type=("." | ".*") selector=Selector? | type=(".@" | ".^") (=> selector=Selector)?) |

// Bracket Selectors
{BracketSelector.data=current} (type="[*" selector=Selector | type=("[@" | "[^") (selector=Selector)?) "]" |
{IndexSelector.data=current} "[" indexSelector=SignedInteger "]" |
{RangeSelector.data=current} "[" start=SignedInteger ".." end=SignedInteger "]" |
{DynamicSelector.data=current} "[" "(" expression=Expression ")" "]" |
{FilterSelector.data=current} "[" "?" "(" expression=FunctionExpression ")" "]" |
{ValueSelector.data=current} "[" selector=Selector "]")* => ({ValueModifier.data=current} type=("?" | "!"))?);

DeclaredNamespace:
name=Identifier "#";

Attributes:
{Attributes} "@(" (attributes+=AttributeElement ("," attributes+=AttributeElement)*)? ")";

/*
* Attributes
*/
AttributeElement:
=> ConditionalAttribute | Attribute | EnclosedExpression;

Attribute:
namespace=DeclaredNamespace? name=Identifier ":" value=Expression;

ConditionalAttribute:
"(" namespace=DeclaredNamespace? name=Identifier ":" value=Expression ")" "when" condition=Expression;

/*
* Literal Object and Expression
*/
SingleKeyValuePairObj:
element=KeyValuePair;

/*
* Expressions
*/
Expression:
ConditionalExpression;

ConditionalExpression returns Expression:
FunctionExpression (-> ({When.trueStatement=current} "when" |
{Unless.trueStatement=current} "unless") condition=FunctionExpression "otherwise" falseStatement=FunctionExpression)*;

FunctionExpression returns Expression:
ReplaceExpression (-> ({MapObjectLambda.element=current} "mapObject" |
{MapLambda.element=current} "map" |
{PluckLambda.element=current} "pluck" |
{FilterLambda.element=current} "filter" |
{ReduceLambda.element=current} "reduce" |
{GroupByLambda.element=current} "groupBy" |
{OrderByLambda.element=current} "orderBy" |
{FindLambda.element=current} "find" |
{ScanLambda.element=current} "scan" |
{MatchLambda.element=current} "match" |
{DistinctByLambda.element=current} "distinctBy") function=FunctionDefintion)*;

ReplaceExpression returns Expression:
OrExpression (-> ({ReplaceWith.element=current} "replace") searchValue=OrExpression "with"
function=FunctionDefintion)*;

OrExpression returns Expression:
AndExpression (-> ({Or.left=current} "or") right=AndExpression)*;

AndExpression returns Expression:
EqualityExpression (-> ({And.left=current} "and") right=EqualityExpression)*;

EqualityExpression returns Expression:
IsExpression (-> ({Equal.left=current} "==" | {NotEqual.left=current} "!=") right=IsExpression)*;

IsExpression returns Expression:
StringExpression (-> ({Is.element=current} "is")
type=Type)*;

StringExpression returns Expression:
GreaterThanExpression (-> ({StartsWith.data=current} "startsWith" |
{EndsWith.data=current} "endsWith" |
{Matches.data=current} "matches" |
{Contains.data=current} "contains") filter=GreaterThanExpression)*;

GreaterThanExpression returns Expression:
LessThanExpression (-> ({GreaterThan.left=current} ">" | {GreaterThanEqual.left=current} ">=")
right=LessThanExpression)*;

LessThanExpression returns Expression:
AdditionSubtractionExpression (-> ({LessThan.left=current} "<" | {LessThanEqual.left=current} "<=")
right=AdditionSubtractionExpression)*;

AdditionSubtractionExpression returns Expression:
AppendExpression (-> ({Plus.left=current} "+" | {Minus.left=current} "-") right=AppendExpression)*;

AppendExpression returns Expression:
RightLeftExpression (-> ({Append.data=current} "++" |
{Remove.data=current} "--") element=RightLeftExpression)*;

RightLeftExpression returns Expression:
MultiplicationDivisionExpression (-> ({RightShift.left=current} ">>" | {Zip.left=current} "zip")
right=MultiplicationDivisionExpression)*;

MultiplicationDivisionExpression returns Expression:
SplitByExpression (-> ({Multiply.left=current} "*" | {Division.left=current} "/") right=SplitByExpression)*;

SplitByExpression returns Expression:
JoinByExpression (-> ({SplitBy.data=current} "splitBy") separator=JoinByExpression)*;

JoinByExpression returns Expression:
AsExpression (-> ({JoinBy.data=current} "joinBy") separator=AsExpression)*;

AsExpression returns Expression:
DefaultValueExpression (-> ({As.element=current} "as")
type=DefaultValueExpression)*;

DefaultValueExpression returns Expression:
Prefixed (-> ({DefaultValue.data=current} "default") default=Prefixed)?;

Prefixed returns Expression:
({UnaryMinus} "-" |
{Negated} "not" |
{SizeOf} "sizeOf" |
{Flatten} "flatten" |
{ValuesOf} "valuesOf" |
{KeysOf} "keysOf" |
{Trim} "trim" |
{Sum} "sum" |
{Average} "avg" |
{Using} "using" "(" parameters+=VariableDefintion ("," parameters+=VariableDefintion)* ")" |
{Upper} "upper" |
{Lower} "lower" |
{Capitalize} "capitalize" |
{Camelize} "camelize" |
{Dasherize} "dasherize" |
{Ordinalize} "ordinalize" |
{Pluralize} "pluralize" |
{Singularize} "singularize" |
{Underscore} "underscore" |
{Typeof} "typeOf" |
{Max} "max" |
{Min} "min" |
{Unzip} "unzip") expression=Postfix |
Postfix;

Postfix returns Expression:
Atomic => (
// We only dispatch one specific action for each selector because dispatching different actions consume a lot of time during the generation of EMF model, lexer and parser

// Descendant Selectors
{DescendantSelector.data=current} type=(".." | "..@" | "..^") (=> selector=Selector)? |

// Dot Selectors
{DotSelector.data=current} (type=("." | ".*") selector=Selector? | type=(".@" | ".^") (=> selector=Selector)?) |

// Bracket Selectors
{BracketSelector.data=current} (type="[*" selector=Selector | type=("[@" | "[^") (selector=Selector)?) "]" |
{IndexSelector.data=current} "[" indexSelector=SignedInteger "]" |
{RangeSelector.data=current} "[" start=SignedInteger ".." end=SignedInteger "]" |
{DynamicSelector.data=current} "[" "(" expression=Expression ")" "]" |
{FilterSelector.data=current} "[" "?" "(" expression=FunctionWithoutParameters ")" "]" |
{ValueSelector.data=current} "[" selector=Selector "]")* => ({ValueModifier.data=current} type=("?" | "!"))?;

Atomic returns Expression:
=> SingleKeyValuePairObj | LiteralValue | => FunctionCall | Variable | MultipleKeyValuePairObj | => Range | => Array | => Type | EnclosedExpression;

EnclosedExpression returns Expression:
"(" Expression ")";

Selector:
namespace=DeclaredNamespace? name=(StringLiteral | Keyword);

/*
* Functions
*/
FunctionDefintion:
=> ("(" "(" (parameters+=FunctionParameter ("," parameters+=FunctionParameter)*)? ")" "->"
expression=ReplaceExpression ")") | => (parameters+=FunctionParameter "->" expression=ReplaceExpression) |
expression=FunctionWithoutParameters;

FunctionWithoutParameters:
ReplaceExpression;

FunctionLiteral:
"(" (parameters+=FunctionParameter ("," parameters+=FunctionParameter)*)? ")" "->" expression=Expression;

FunctionCall:
Variable "(" (parameters+=Expression ("," parameters+=Expression)*)? ")";

/*
* Variables
*/
Variable:
DoubleDollarVariable | VariableReference | DollarVariable;

VarDefinition:
VariableDirective | InputDirective | FunctionParameter | VariableDefintion;

VariableReference:
{VariableReference} reference=Identifier;

DollarVariable:
{DollarVariable} "$";

DoubleDollarVariable:
{DoubleDollarVariable} "$$";

FunctionParameter returns Expression:
name=Identifier ("=" initialValue=Expression)?;

VariableDefintion returns Expression:
{VariableDefintion} name=Identifier "=" initialValue=Expression;

/*
* Literals
*
*/
LiteralValue:
{BooleanLiteral} value=BooleanLiteral | {NullLiteral} value=NullLiteral | {QuotedStringLiteral} value=QuotedStringLiteral |
{RealLiteral} value=RealLiteral | {IntegerLiteral} value=IntegerLiteral | {AnyDateLiteral} value=AnyDateLiteral |
{AnyRegexLiteral} value=AnyRegexLiteral;

/*
* Simple Types
*/
NullLiteral:
"null";

BooleanLiteral returns ecore::EBoolean:
"true" | "false";

StringLiteral:
QuotedStringLiteral | Identifier;

QuotedStringLiteral:
STRING;

NamespacePrefix:
NAMESPACE_PREFIX;

NamespaceUri:
NAMESPACE_URI;

SignedInteger returns ecore::EBigDecimal hidden():
("-")? INT;

IntegerLiteral returns ecore::EBigDecimal:
INT;

RealLiteral returns ecore::EBigDecimal:
INT "." INT | INT "." EXPONENTIAL | EXPONENTIAL "." INT | EXPONENTIAL "." EXPONENTIAL;

AnyDateLiteral returns ecore::EString:
ANY_DATE;

AnyRegexLiteral returns ecore::EString:
ANY_REGEX;

// TODO (ldebello) List of all possible keywords that we want to use as common strings
Keyword:
"true" | "false" | "not";

Identifier:
UNQUOTED_STRING;

/*
* Complex Types
*/
Range:
"[" start=SignedInteger ".." end=SignedInteger "]";

Array:
{Array} "[" (elements+=ArrayElement ("," elements+=ArrayElement)*)? "]";

ArrayElement:
=> ConditionalArrayElement | Expression;

ConditionalArrayElement:
"(" expression=Expression ")" "when" condition=Expression;

Type:
{Type} ':' name=Identifier (=> schema=Schema?);

/*
* Enums
*/
enum DataType:
APPLICATION_DFL="application/weave" | APPLICATION_JSON="application/json" | APPLICATION_XML="application/xml" |
TEXT_XML="text/xml" | APPLICATION_CSV="application/csv" | APPLICATION_JAVA="application/java";

/*
* Terminals
*/
terminal STRING:
'"' ('\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\' | '"'))* '"'? |
"'" ('\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\' | "'"))* "'"?;

terminal INT returns ecore::EBigDecimal:
DIGIT+;

terminal EXPONENTIAL returns ecore::EBigDecimal:
DIGIT+ (('e' | 'E') DIGIT+);

terminal ANY_DATE:
'|'->'|';

terminal ANY_REGEX:
'/'->'/';

terminal UNQUOTED_STRING:
!NON_QUOTED_STRING_START !(NON_QUOTED_STRING_END)*;

terminal WS:
(' ' | '\t')+;

terminal SL_COMMENT:
'#' !(LINE_BREAK)*;

terminal EOL:
LINE_BREAK;

terminal NAMESPACE_PREFIX:
('a'..'z' | 'A'..'Z' | '0'..'9' | '_' | '-')*;

terminal NAMESPACE_URI:
('a'..'z' | 'A'..'Z' | '0'..'9' | '_' | '-' | '.' | ':' | '/')*;

terminal fragment LINE_BREAK:
('\r' | '\n');

terminal fragment DIGIT:
('0'..'9');

terminal fragment NON_QUOTED_STRING_START:
'"' | "'" | '0'..'9' | '!' | '#' | '$' | '(' | ')' | '*' | '+' | ',' | '-' | '.' | '/' | ':' | '<' | '=' | '>' | '?' |
'[' | ']' | '{' | '}' | '|' | '%' | '^' | '@' | '\r' | '\n' | ' ' | '\t';

terminal fragment NON_QUOTED_STRING_END:
('!' | '#' | '$' | '(' | ')' | '*' | ',' | '.' | ':' | '<' | '=' | '>' | '?' | '[' | ']' | '{' | '}' | '|' | '%'
| '^' | '\r' | '\n' | ' ' | '\t');

Also I am using a custom lexer to add some context in order to resolve some ambiguities.

I really appreciate your help or any tip related to the concepts of Content Assist Context or Follow Elements.

After debugging the code I understand that I will need to do one of the following things:

1- Relax my grammar to allow invalidate grammar in order to receive the complete node as valid and then add validation using the validation module (To be honest I am not really fond of this, but it seems the way to go)
2- Adding more specific logic to create different context using the offset to parse the current content (I think that this could be a solution but after changing that code I am sure that I will got a lot of error or dispatching the content assist several time)

Maybe you can tell me if my options are the right ones or if I am totally wrong in order to find the best solution.

Thanks in advance for you time and comments

Regards,
Luis

Attachment: Grammar
(Size: 12.72KB, Downloaded 93 times)

[Updated on: Fri, 29 May 2015 13:58]

Report message to a moderator

Re: Content Assist Contexts [message #1697014 is a reply to message #1696936]

Sun, 31 May 2015 07:17

Luis De Bello

Messages: 95
Registered: January 2015

Member

Hi Christian/Sven

I think that I found a possible fix for my issue, however I think this is a more generic thing, maybe you can confirm this.
After debugging the content assist logic I realised that there is another parser for the content assist and this is provided with a "ObservableXtextTokenStream", but in the class "AbstractContentAssistParser" there is one method called "public Collection<FollowElement> getFollowElements(FollowElement element)" which creates a "ObservableXtextTokenStream" without setting the hidden tokens. After setting the hidden tokens, the code start calling the methods from the proposal providers

Source Code:

AbstractInternalContentAssistParser parser = createParser();
parser.setUnorderedGroupHelper(getUnorderedGroupHelper().get());
parser.getUnorderedGroupHelper().initializeWith(parser);
final Iterator<LookAheadTerminal> iter = element.getLookAheadTerminals().iterator();
ObservableXtextTokenStream tokens = new ObservableXtextTokenStream(new TokenSource() {

    @Override
    public Token nextToken() {
        if (iter.hasNext()) {
            LookAheadTerminal lookAhead = iter.next();
            return lookAhead.getToken();
        }
        return Token.EOF_TOKEN;
    }

    @Override
    public String getSourceName() {
        return "LookAheadTerminalTokenSource";
    }
}, parser);

// Here I added the line following line
// tokens.setInitialHiddenTokens(getInitialHiddenTokens());

parser.setTokenStream(tokens);

Do you know if this was done in this way for an specific reason or it is a bug?

Thanks in advance

Regards,
Luis

Report message to a moderator

Re: Content Assist Contexts [message #1697017 is a reply to message #1697014]

Sun, 31 May 2015 11:31

Sebastian Zarnekow

Messages: 3118
Registered: July 2009

Senior Member

Hi,

yes, that looks like a bug though initialHiddenTokens is probably not
what we want to have there but the complete hidden tokens stack that was
valid when the follow element was produced. Could you please file a ticket?

Best,
Sebastian
--
Looking for professional support for Xtext, Xtend or Eclipse Modeling?
Find help at http://xtext.itemis.com or xtext(@)itemis.com
Blog: zarnekow.blogspot.com
Twitter: @szarnekow
Google+: https://www.google.com/+SebastianZarnekow

Report message to a moderator

Re: Content Assist Contexts [message #1697020 is a reply to message #1697017]

Sun, 31 May 2015 13:54

Luis De Bello

Messages: 95
Registered: January 2015

Member

Hi Sebastian,

Thanks for your prompt response, I will file a ticket.
I thought about different rules with different hidden tokens but I was not sure how to solve that case, anyway please correct me if I am wrong but as long as you don't overriden your hidden token the initialtokens is fine, is that ok?

Thanks for your time

Best regards,
Luis

Report message to a moderator

Re: Content Assist Contexts [message #1697024 is a reply to message #1697020]

Sun, 31 May 2015 15:35

Sebastian Zarnekow

Messages: 3118
Registered: July 2009

Senior Member

Yes, that's right.

Best,
Sebastian
--
Looking for professional support for Xtext, Xtend or Eclipse Modeling?
Find help at http://xtext.itemis.com or xtext(@)itemis.com
Blog: zarnekow.blogspot.com
Twitter: @szarnekow
Google+: https://www.google.com/+SebastianZarnekow

Report message to a moderator