Home » Eclipse Projects » Dynamic Languages Toolkit (DLTK) » ASTNode - Expression/Statement suggestion
| ASTNode - Expression/Statement suggestion [message #2218] |
Mon, 23 April 2007 14:43  |
Eclipse User |
|
|
|
I spent the last few days trying to figure out why the SearchEngine doesn't return certain specific
types (for instance ActiveRecord::Migration). I realized that if I browse to a type in the
ScriptExplorer and open one of the types that the SeachEngine doesn't return there is no structure.
Then I realized that JRubySourceParser was eating exceptions (intentionally), so to aid in debugging
I allowed the exceptions. Seems like all the problems occurred where an ast was expecting a child
AST to be an expression but it was actually a Statement of some kind (like a RubyCaseStatement or an
IfStatement).
I added some patches to RubyASTBuildVisitor which gets around these problems but it points out a
basic problem with the structure of the tree. The ASTNode tree has Statement as a child of ASTNode.
Statement has many children of it's own, one of which is Expression. The problem is that what is a
statement and an expression is different for each language. In Ruby almost everything is an
expression, eg. case statements, conditional statements, methods etc. This makes building the Ruby
version of the DLTK AST tree very difficult when Expression is a child of Statement.
I suggest a different approach. Remove Statement and Expression completely from the tree (ie flatten
the tree). Have most AST's subclass ASTNode directly except in specific cases such as a list node
(although another approach would be where ASTNode is either a single node or a list of nodes and
each node adapts what is returned). This would remove the need to perform any casting in the
building of each languages AST tree. I would add the knowledge of whether a node is an expression or
not in ASTNode and each language builder would let each node know what type of node it is as it's
built. This allows the target language to define what is an expression, not a predifined inhertiance
hierachy.
The AST structure should be defined by the dynamic relationship of nodes as they are built not in a
static hierarchy tree. Otherwise you get structure being defined in two places, the hierachy and the
dynamic relationship.
Thanks
Mark
|
|
|
| Re: ASTNode - Expression/Statement suggestion [message #2248 is a reply to message #2218] |
Wed, 25 April 2007 09:22  |
| Eclipse User |
|
|
|
Originally posted by: mikhail.kalugin.xored.com
Hi, Mark.
We have discussed in details this problem inside our team today. Frankly
speaking we had already thought about solution some months ago, but
somehow it has left unsolved till this moment.
Anyway, we are fully agree with a suggested solution. We decided to
implement it in practice, and in the nearest future, I think, problem
will be solved as you suggested.
Thanks a lot for your help.
--
Mikhail Kalugin
Mark Howe wrote:
> I spent the last few days trying to figure out why the SearchEngine
> doesn't return certain specific types (for instance
> ActiveRecord::Migration). I realized that if I browse to a type in the
> ScriptExplorer and open one of the types that the SeachEngine doesn't
> return there is no structure. Then I realized that JRubySourceParser was
> eating exceptions (intentionally), so to aid in debugging I allowed the
> exceptions. Seems like all the problems occurred where an ast was
> expecting a child AST to be an expression but it was actually a
> Statement of some kind (like a RubyCaseStatement or an IfStatement).
>
> I added some patches to RubyASTBuildVisitor which gets around these
> problems but it points out a basic problem with the structure of the
> tree. The ASTNode tree has Statement as a child of ASTNode. Statement
> has many children of it's own, one of which is Expression. The problem
> is that what is a statement and an expression is different for each
> language. In Ruby almost everything is an expression, eg. case
> statements, conditional statements, methods etc. This makes building the
> Ruby version of the DLTK AST tree very difficult when Expression is a
> child of Statement.
>
> I suggest a different approach. Remove Statement and Expression
> completely from the tree (ie flatten the tree). Have most AST's subclass
> ASTNode directly except in specific cases such as a list node (although
> another approach would be where ASTNode is either a single node or a
> list of nodes and each node adapts what is returned). This would remove
> the need to perform any casting in the building of each languages AST
> tree. I would add the knowledge of whether a node is an expression or
> not in ASTNode and each language builder would let each node know what
> type of node it is as it's built. This allows the target language to
> define what is an expression, not a predifined inhertiance hierachy.
>
> The AST structure should be defined by the dynamic relationship of nodes
> as they are built not in a static hierarchy tree. Otherwise you get
> structure being defined in two places, the hierachy and the dynamic
> relationship.
>
> Thanks
> Mark
|
|
|
| Re: ASTNode - Expression/Statement suggestion [message #2278 is a reply to message #2248] |
Wed, 25 April 2007 22:34 |
| Eclipse User |
|
|
|
Kool! Thanks Mikhail
Mark
Mikhail Kalugin wrote:
> Hi, Mark.
>
> We have discussed in details this problem inside our team today. Frankly
> speaking we had already thought about solution some months ago, but
> somehow it has left unsolved till this moment.
>
> Anyway, we are fully agree with a suggested solution. We decided to
> implement it in practice, and in the nearest future, I think, problem
> will be solved as you suggested.
>
> Thanks a lot for your help.
>
> --
> Mikhail Kalugin
>
> Mark Howe wrote:
>> I spent the last few days trying to figure out why the SearchEngine
>> doesn't return certain specific types (for instance
>> ActiveRecord::Migration). I realized that if I browse to a type in the
>> ScriptExplorer and open one of the types that the SeachEngine doesn't
>> return there is no structure. Then I realized that JRubySourceParser
>> was eating exceptions (intentionally), so to aid in debugging I
>> allowed the exceptions. Seems like all the problems occurred where an
>> ast was expecting a child AST to be an expression but it was actually
>> a Statement of some kind (like a RubyCaseStatement or an IfStatement).
>>
>> I added some patches to RubyASTBuildVisitor which gets around these
>> problems but it points out a basic problem with the structure of the
>> tree. The ASTNode tree has Statement as a child of ASTNode. Statement
>> has many children of it's own, one of which is Expression. The problem
>> is that what is a statement and an expression is different for each
>> language. In Ruby almost everything is an expression, eg. case
>> statements, conditional statements, methods etc. This makes building
>> the Ruby version of the DLTK AST tree very difficult when Expression
>> is a child of Statement.
>>
>> I suggest a different approach. Remove Statement and Expression
>> completely from the tree (ie flatten the tree). Have most AST's
>> subclass ASTNode directly except in specific cases such as a list node
>> (although another approach would be where ASTNode is either a single
>> node or a list of nodes and each node adapts what is returned). This
>> would remove the need to perform any casting in the building of each
>> languages AST tree. I would add the knowledge of whether a node is an
>> expression or
>> not in ASTNode and each language builder would let each node know what
>> type of node it is as it's built. This allows the target language to
>> define what is an expression, not a predifined inhertiance hierachy.
>>
>> The AST structure should be defined by the dynamic relationship of
>> nodes as they are built not in a static hierarchy tree. Otherwise you
>> get structure being defined in two places, the hierachy and the
>> dynamic relationship.
>>
>> Thanks
>> Mark
|
|
|
| Re: ASTNode - Expression/Statement suggestion [message #2396 is a reply to message #2278] |
Fri, 04 May 2007 07:10 |
| Eclipse User |
|
|
|
Originally posted by: mikhail.kalugin.xored.com
Hi,
We have done refactoring and now ruby's AST is flat(ruby ast nodes
directly subclasses ASTNode or ASTListNode). You can update from cvs and
see the results.
Statement and Expression classes has been left in DLTK Core to save
code for Tcl and Python, where they are still used, but Core itself
and Ruby should not use them.
Mark Howe wrote:
> Kool! Thanks Mikhail
>
> Mark
>
>
> Mikhail Kalugin wrote:
>> Hi, Mark.
>>
>> We have discussed in details this problem inside our team today.
>> Frankly speaking we had already thought about solution some months
>> ago, but somehow it has left unsolved till this moment.
>>
>> Anyway, we are fully agree with a suggested solution. We decided to
>> implement it in practice, and in the nearest future, I think, problem
>> will be solved as you suggested.
>>
>> Thanks a lot for your help.
>>
>> --
>> Mikhail Kalugin
>>
>> Mark Howe wrote:
>>> I spent the last few days trying to figure out why the SearchEngine
>>> doesn't return certain specific types (for instance
>>> ActiveRecord::Migration). I realized that if I browse to a type in
>>> the ScriptExplorer and open one of the types that the SeachEngine
>>> doesn't return there is no structure. Then I realized that
>>> JRubySourceParser was eating exceptions (intentionally), so to aid in
>>> debugging I allowed the exceptions. Seems like all the problems
>>> occurred where an ast was expecting a child AST to be an expression
>>> but it was actually a Statement of some kind (like a
>>> RubyCaseStatement or an IfStatement).
>>>
>>> I added some patches to RubyASTBuildVisitor which gets around these
>>> problems but it points out a basic problem with the structure of the
>>> tree. The ASTNode tree has Statement as a child of ASTNode. Statement
>>> has many children of it's own, one of which is Expression. The
>>> problem is that what is a statement and an expression is different
>>> for each language. In Ruby almost everything is an expression, eg.
>>> case statements, conditional statements, methods etc. This makes
>>> building the Ruby version of the DLTK AST tree very difficult when
>>> Expression is a child of Statement.
>>>
>>> I suggest a different approach. Remove Statement and Expression
>>> completely from the tree (ie flatten the tree). Have most AST's
>>> subclass ASTNode directly except in specific cases such as a list
>>> node (although another approach would be where ASTNode is either a
>>> single node or a list of nodes and each node adapts what is
>>> returned). This would remove the need to perform any casting in the
>>> building of each languages AST tree. I would add the knowledge of
>>> whether a node is an expression or
>>> not in ASTNode and each language builder would let each node know
>>> what type of node it is as it's built. This allows the target
>>> language to define what is an expression, not a predifined
>>> inhertiance hierachy.
>>>
>>> The AST structure should be defined by the dynamic relationship of
>>> nodes as they are built not in a static hierarchy tree. Otherwise you
>>> get structure being defined in two places, the hierachy and the
>>> dynamic relationship.
>>>
>>> Thanks
>>> Mark
|
|
|
Goto Forum:
Current Time: Sun May 11 15:40:58 EDT 2025
Powered by FUDForum. Page generated in 0.03666 seconds |
Home