Lexer mixes some inputs even though it is set to non-greedy? [message #1841801] |
Sun, 30 May 2021 17:40 |
M D Messages: 33 Registered: January 2021 |
Member |
|
|
I have created a simple grammar, where a value can be a refType:
RefType: 'Aggregate' | 'Composite' | 'Association';
When the value is set to Aggregate or Association, it does not give an error. However, when Composite is written, it gives "no viable alternative at character 'o'. In the DSL it happens at "value": "Composite" . It is the second to last parser rule named ClassReferences in the grammar, at this exact line: '"value"' ':' '"' typeValue = RefType '"'.
I have had a similar error before which was fixed by setting lexer to being greedy:
parserGenerator = {
options = {
backtrackLexer = true
}
}
This however doesn't solve it this time, and I don't really get how changing Composite to Aggregate or Association fixes the problem.
The grammar:
Model:
{Model}
'[' tables += Table? (',' tables += Table)* ']'
;
Table:
ClassTable | ClassReferencesTable
;
ClassTable:
{ClassTable}
'{'
'"Name"' ':' '"Class"' ','
'"Table"' ':' '['
class += Class? (',' class += Class)*
']'
'}'
;
Class:
'{'
'"Name"' ':' '{'
'"column"' ':' nameColumn = INT ','
'"row"' ':' nameRow = INT ','
'"value"' ':' name = STRING
'}' (',')?
'"References"' ':' '['
((references += ClassReferences (',' references += ClassReferences)*) | (references += ClassReferencesReference (',' references += ClassReferencesReference)*))
']'
'}'
;
ClassReferencesTable:
{ClassReferencesTable}
'{'
'"Name"' ':' '"ClassReferences"' ','
'"Table"' ':' '['
references += ClassReferences? (',' references += ClassReferences)*
']'
'}'
;
ClassReferencesReference:
'{'
'"Name"' ':' '{'
'"column"' ':' column = INT ','
'"row"' ':' row = INT ','
'"value"' ':' name = [ClassReferences|STRING]
'}'
'}'
;
ClassReferences:
'{'
'"Type"' ':' '{'
'"column"' ':' typeColumn = INT ','
'"row"' ':' typeRow = INT ','
'"value"' ':' '"' typeValue = RefType '"'
'}' (',')?
'"To"' ':' '{'
'"column"' ':' toColumn = INT ','
'"row"' ':' toRow = INT ','
'"value"' ':' toValue = [Class|STRING]
'}'
'}'
;
RefType: 'Aggregate' | 'Composite' | 'Association';
terminal NULL: 'null';
terminal ID: '^'?('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*;
terminal STRING: '"OKZVVTSPKHOVYSMU' -> 'SQPSUQMWUPQSBXDT"';
terminal INT returns ecore::EInt: ('0'..'9')+;
terminal FLOAT: '-'? INT? '.' INT (('E'|'e') '-'? INT)?;
terminal BOOLEAN: 'true' | 'false';
terminal ML_COMMENT: '/*' -> '*/';
terminal SL_COMMENT: '//' !('\n'|'\r')* ('\r'? '\n')?;
terminal WS: (' '|'\t'|'\r'|'\n')+;
terminal ANY_OTHER: .;
The DSL:
[
{
"Name": "Class",
"Table": [
{
"Name": {
"column": 0,
"row": 4,
"value": "OKZVVTSPKHOVYSMUSchoolSQPSUQMWUPQSBXDT"
},
"References": [
{
"Type": {
"column": 1,
"row": 4,
"value": "Composite"
},
"To": {
"column": 2,
"row": 4,
"value": "OKZVVTSPKHOVYSMUClassRoomSQPSUQMWUPQSBXDT"
}
}
]
},
{
"Name": {
"column": 0,
"row": 5,
"value": "OKZVVTSPKHOVYSMUClassRoomSQPSUQMWUPQSBXDT"
},
"References": [
{
"Type": {
"column": 1,
"row": 5,
"value": "Aggregate"
},
"To": {
"column": 2,
"row": 5,
"value": "OKZVVTSPKHOVYSMUSchoolSQPSUQMWUPQSBXDT"
}
}
]
}
]
}
]
[Updated on: Sun, 30 May 2021 19:42] Report message to a moderator
|
|
|
|
Re: Lexer mixes some inputs even though it is set to greedy? [message #1841803 is a reply to message #1841802] |
Sun, 30 May 2021 19:07 |
|
maybe you can also try to customize the lexer to be non greedy instead of backtracking only.
have zero idea if this has any sideeffects
parserGenerator =CustomXtextAntlrGeneratorFragment2 {
options= {
backtrackLexer=true
}
}
class CustomXtextAntlrGeneratorFragment2 extends XtextAntlrGeneratorFragment2 {
protected def compileLexerOptions(Grammar it, AntlrOptions options) '''
«IF options.backtrackLexer»
options {
backtrack=true;
memoize=true;
greedy=false;
}
«ENDIF»
'''
}
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
|
|
|
|
|
|
|
|
|
|
|
|
Powered by
FUDForum. Page generated in 0.04765 seconds