Lua xText Grammar: Distinguish between variable reference and declaration [message #1870659] |
Wed, 04 September 2024 17:26 |
Juan S Messages: 6 Registered: September 2024 |
Junior Member |
|
|
Hello,
I am currently working on an xText project to describe the Lua programming language.
My grammar is based on the syntax given in the Lua 5.2 Reference Manual [1]. I have already implemented the grammar in a way that seems to parse lua files correctly.
This is a shortened version of that grammar (leaving out (most of ) what is not related to my problem):
Block:
{Block} (stats+=Stat (=>';')?)*
;
Stat returns Stat:
AssignmentOrFunctionCall
;
AssignmentOrFunctionCall returns Stat:
PrefixExp (
({FunctionCall.functionCall=current}) |
({Assignment.vars+=current} (',' vars+=PrefixExp)* =>'=' expList=ExpList)
)
;
// This rule is only used to stay close to the Lua syntax reference documentation, where prefixexp is a rule
PrefixExp returns Exp:
SuffixExp
;
SuffixExp returns Exp:
Var =>(
':' {MemberFuncCall.object=current} name=[Var|LUA_NAME] args=Args |
{FuncCall.object=current} args=Args |
'[' {VarArrayAccess.array=current} index=Exp ']' |
'.' {VarMemberAccess.object=current} name=[Var|LUA_NAME]
)*
;
Var returns Exp:
'(' Exp ')' |
{Var} name=[Var|LUA_NAME] // I'm attempting to implement cross-references and scoping with this name=[Var|LUA_NAME]
;
Args:
{ParamArgs} '(' (params=ExpList)? ')' |
//TableConstructor | ommitted for simplicity of example
{LiteralStringArg} str=LUA_STRING
;
Exp:
ExpPrimary
;
ExpPrimary returns Exp:
PrefixExp |
{ExpNumberLiteral} value=NUMBER_LITERAL |
{ExpStringLiteral} value=LUA_STRING
// {ExpNil} 'nil' | {ExpFalse} 'false' | ...
;
ExpList:
exps+=Exp ( =>',' exps+=Exp)*
;
The next step would be to implement cross-references and scoping, which is where my problem arises: Vars can be referenced (e.g. when on the left side of an assignment) but also need to reference (e.g. when occuring on the right side of an asignment). These constructs are seem to be syntactically equivalent in Lua.
My first idea (based on a stackoverflow post [2]) was to use the ILinkingDiagnosticMessageProvider.getUnresolvedProxyMessage to ignore the messages in the context of left-hand sides of assignments (which works) and to somehow programatically distinguish between referencing and referenced Vars (which I don't know how to do).
I did not find any solutions to this problem. I found a post that comes close on stackoverflow [3], but there a delcaration is preceded by some keyword (e.g. "int a = 5"), whereas in Lua there is no such keyword to distinguish the lhs and rhs of an assignment.
I hope this description gets my problem across, feel free to ask if anything is unclear.
Thank you in advance
PS: I could not add the links because this is my first post.
Edit (adding links):
[1] Lua 5.2 Reference Manual
[2] stackoverflow post
[3] stackoverflow post with similar problem
[Updated on: Wed, 04 September 2024 18:54] Report message to a moderator
|
|
|
|
Re: Lua xText Grammar: Distinguish between variable reference and declaration [message #1870671 is a reply to message #1870665] |
Wed, 04 September 2024 18:51 |
Juan S Messages: 6 Registered: September 2024 |
Junior Member |
|
|
Sorry, I find it difficult to describe the problem concisely.
Basically the Lua syntax allows for the same expressions on the right-hand side and left-hand side of an assignment, given by the following rules from the reference:
stat ::= varlist '=' explist
var ::= Name | prefixexp '[' exp ']' | prefixexp '.' Name
prefixexp ::= var | functioncall | '(' exp ')'
functioncall ::= prefixexp args | prefixexp ':' Name args
Now in my xText grammar, after removing left-recursion and ambiguities these rules are described by (PrefixExp,) SuffixExp and Var (LUA_NAME from my grammar corresponds to Name from the reference). But then it is possible for a PrefixExp to reference another PrefixExp, i.e. some PrefixExp are "Referenceable" and other "Referencing".
Now I don't know how to implement cross-references in this case. As an example:
var = 1 // var should be referenceable, i.e. have a QualifiedName
a = var // var should reference the previously declared var
My idea was to use name=[Var|LUA_NAME] as a cross-reference and a QualifiedName at the same time, deciding after the parsing phase if "name" is an attribute or a reference. Unfortunately, I was unable to implement this idea as of now.
[Updated on: Wed, 04 September 2024 20:46] Report message to a moderator
|
|
|
Re: Lua xText Grammar: Distinguish between variable reference and declaration [message #1870707 is a reply to message #1870671] |
Thu, 05 September 2024 15:58 |
Juan S Messages: 6 Registered: September 2024 |
Junior Member |
|
|
The more I think about this, the more I think it is not possible using xText cross-references (please correct me if I'm wrong):
The problem is that in Lua, there is no syntactical distinction between a variable declaration and a variable reference/access (except for the '=' sign in between and the position of the parsed word on the left or the right of the '=').
So in my case the rule that describes variables would need to reference itself, but the same word that is the reference also needs to be the attribute (in my grammar the name=[Var|LUA_NAME] would at the same time need to be name=LUA_NAME).
This, I think, is not possible with xText.
If that is the case, is there any way to manually implement the Linking?
More specifically: Is it possible to leave the grammar completely reference-free, and manipulate the AST-creation or the Linking phase programatically?
Example:
Using the example from my previous post (and my grammar adjusted so that all name=[Var|LUA_NAME] references are now attributes name=LUA_NAME):
var = 1 // creates the EObject Var (now with an attribute "name"=var)
a = var // creates two EObjects Var (with "name"=a and "name"=var)
Will return an AST like this:
Block
Assignment
Var (name: var) // 1. Var EObject with name "var" (should be Referenceable)
ExpList
ExpNumberLiteral (value: 1.0)
Assignment
Var (name: a)
ExpList
Var (name: var) // 2. Var EObject with name "var" (should be Referencing)
Now my idea for the manual linking would be to check if the Var is part of an ExpList (e.g. 2. Var EObject), and then replace it with another EObject that contains an EReference "name" instead of the EAttribute "name" (this reference would then need to contain the LinkingText from the EAttribute or smth. like that).
Is this something that is possible, and if so how would one go about it?
[Updated on: Thu, 05 September 2024 15:59] Report message to a moderator
|
|
|
|
Re: Lua xText Grammar: Distinguish between variable reference and declaration [message #1870716 is a reply to message #1870713] |
Thu, 05 September 2024 17:15 |
Juan S Messages: 6 Registered: September 2024 |
Junior Member |
|
|
Not regarding the parser rule, i.e. the words left and right of the '=' are parsed by the same rule. That is why I needed to implement this go-between:
AssignmentOrFunctionCall returns Stat:
PrefixExp (
({FunctionCall.functionCall=current}) | // a functioncall is parsed by the PrefixExp rule
({Assignment.vars+=current} (',' vars+=PrefixExp)* =>'=' expList=ExpList) // so are assignments, and some Expressions: The ExpList contains Expressions, which can be resolved to PrefixExps
)
;
A PrefixExp can describe a lhs word, a rhs word, function arguments, basically a lot of parts of the Lua language. Since no ambiguities are allowed in the grammar, the same rule applies to any word that is represented by a PrefixExp (something like "array[0].member(arg1, arg2)["temp"] = "hello world" is allowed in Lua, and any word on the lhs could also appear on the rhs).
|
|
|
|
|
|
Re: Lua xText Grammar: Distinguish between variable reference and declaration [message #1870788 is a reply to message #1870725] |
Mon, 09 September 2024 10:17 |
Juan S Messages: 6 Registered: September 2024 |
Junior Member |
|
|
First of all, thank you very much for your help up to this point!
I did not know that we can use two rules with the same syntax at different positions, so that solves the assignment problem (I just copy-pasted the PrefixExp, SuffixExp, and Var rules, and used the copy on the rhs).
Unfortunately, this does still not solve the problem completely: I'm still not able to distinguish Functioncalls and assignment variables in my grammar in a way that let's me assign references to one (the FunctionCall) and a name to the other (the assignment variable).
In Lua, the syntax for a FunctionCall and an assignment variable are very similar, described in my current grammar by the PrefixExp rule and distinguished by the AssignmentOrFunctionCall rule:
AssignmentOrFunctionCall returns Stat:
PrefixExp (
({FunctionCall.functionCall=current}) |
({Assignment.vars+=current} (',' vars+=PrefixExp)* =>'=' expList=ExpList)
)
;
// This rule is only used to stay close to the Lua syntax reference documentation, where prefixexp is a rule
PrefixExp returns Exp:
SuffixExp
;
SuffixExp returns Exp:
Var =>(
':' {MemberFuncCall.object=current} name=LUA_NAME args=Args | // denotes functioncall
{FuncCall.object=current} args=Args | // denotes functioncall
'[' {VarArrayAccess.array=current} index=Exp ']' | // denotes variable access
'.' {VarMemberAccess.object=current} name=LUA_NAME // denotes variable access
)*
;
Var returns Exp:
'(' Exp ')' |
{Var} name=LUA_NAME
;
The difference is, that a FunctionCall must end with args, whereas an assignment variable may not end with args (and that assignment variables are followed by an '=' at some point).
That is, both may start with '(' Exp ')' or a LUA_NAME (in the code snippet above defined by the "Var" rule), followed by any number of infixes (the "SuffixExp" rule). Functioncalls end with args (the first two alternatives in the "SuffixExp" rule), while variables end with a table access, member access, or consist of only the LUA_NAME (the third and fourth alternatives in the "SuffixExp" rule and the second alternative in the "Var" rule).
I have tried splitting the PrefixExp rule into two rules in an attempt to distinguish a functioncall and an assignment variable depending on if they end with args (see below), but always end up with a non-LL(*) decision error when compiling the grammar.
Is there any way to implement this destinction?
Attempt to split rule:
Stat returns Stat:
//AssignmentOrFunctionCall // now split into FunctionCall and Assignment
FunctionCall |
Assignment
;
Assignment:
vars+=Var (',' vars+=Var)* =>'=' expList=ExpList
;
// FunctionCall rules need cross-references
FunctionCall returns Exp:
FunctionCallInfix =>(
':' {MemberFuncCall.object=current} ref=[Ref|LUA_NAME] =>args=Args |
{FuncCall.object=current} =>args=Args
)
;
FunctionCallInfix returns Exp:
FunctionCallPrefix =>(
':' {MemberFuncCall.object=current} ref=[Ref|LUA_NAME] args=Args |
{FuncCall.object=current} args=Args |
'[' {FuncCallTableAccess.array=current} index=Exp ']' |
'.' {FuncCallMemberAccess.object=current} ref=[Ref|LUA_NAME]
)*
;
FunctionCallPrefix returns Exp:
'(' Exp ')' |
ref=[Ref|LUA_NAME]
;
// Var rules need a name to reference to
Var returns Var:
-> name=LUA_NAME |
VarInfix => (
=>'[' {VarTableAccess.array=current} index=Exp ']' |
=>'.' {VarMemberAccess.object=current} name=LUA_NAME
)
;
VarInfix returns Var:
VarPrefix =>(
':' {VarMemberFuncCall.object=current} name=LUA_NAME args=Args |
{VarFuncCall.object=current} args=Args |
'[' {VarTableAccess.array=current} index=Exp ']' |
'.' {VarMemberAccess.object=current} name=LUA_NAME
)*
;
VarPrefix returns Var:
'(' Exp ')' |
{Var} name=LUA_NAME
;
|
|
|
|
Powered by
FUDForum. Page generated in 0.07698 seconds