Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Dynamic Languages Toolkit (DLTK) » ASTNode - Expression/Statement suggestion
ASTNode - Expression/Statement suggestion [message #2218] Mon, 23 April 2007 18:43 Go to next message
Mark Howe is currently offline Mark HoweFriend
Messages: 14
Registered: July 2009
Junior Member
I spent the last few days trying to figure out why the SearchEngine doesn't return certain specific
types (for instance ActiveRecord::Migration). I realized that if I browse to a type in the
ScriptExplorer and open one of the types that the SeachEngine doesn't return there is no structure.
Then I realized that JRubySourceParser was eating exceptions (intentionally), so to aid in debugging
I allowed the exceptions. Seems like all the problems occurred where an ast was expecting a child
AST to be an expression but it was actually a Statement of some kind (like a RubyCaseStatement or an
IfStatement).

I added some patches to RubyASTBuildVisitor which gets around these problems but it points out a
basic problem with the structure of the tree. The ASTNode tree has Statement as a child of ASTNode.
Statement has many children of it's own, one of which is Expression. The problem is that what is a
statement and an expression is different for each language. In Ruby almost everything is an
expression, eg. case statements, conditional statements, methods etc. This makes building the Ruby
version of the DLTK AST tree very difficult when Expression is a child of Statement.

I suggest a different approach. Remove Statement and Expression completely from the tree (ie flatten
the tree). Have most AST's subclass ASTNode directly except in specific cases such as a list node
(although another approach would be where ASTNode is either a single node or a list of nodes and
each node adapts what is returned). This would remove the need to perform any casting in the
building of each languages AST tree. I would add the knowledge of whether a node is an expression or

not in ASTNode and each language builder would let each node know what type of node it is as it's
built. This allows the target language to define what is an expression, not a predifined inhertiance
hierachy.

The AST structure should be defined by the dynamic relationship of nodes as they are built not in a
static hierarchy tree. Otherwise you get structure being defined in two places, the hierachy and the
dynamic relationship.

Thanks
Mark
Re: ASTNode - Expression/Statement suggestion [message #2248 is a reply to message #2218] Wed, 25 April 2007 13:22 Go to previous messageGo to next message
Eclipse UserFriend
Originally posted by: mikhail.kalugin.xored.com

Hi, Mark.

We have discussed in details this problem inside our team today. Frankly
speaking we had already thought about solution some months ago, but
somehow it has left unsolved till this moment.

Anyway, we are fully agree with a suggested solution. We decided to
implement it in practice, and in the nearest future, I think, problem
will be solved as you suggested.

Thanks a lot for your help.

--
Mikhail Kalugin

Mark Howe wrote:
> I spent the last few days trying to figure out why the SearchEngine
> doesn't return certain specific types (for instance
> ActiveRecord::Migration). I realized that if I browse to a type in the
> ScriptExplorer and open one of the types that the SeachEngine doesn't
> return there is no structure. Then I realized that JRubySourceParser was
> eating exceptions (intentionally), so to aid in debugging I allowed the
> exceptions. Seems like all the problems occurred where an ast was
> expecting a child AST to be an expression but it was actually a
> Statement of some kind (like a RubyCaseStatement or an IfStatement).
>
> I added some patches to RubyASTBuildVisitor which gets around these
> problems but it points out a basic problem with the structure of the
> tree. The ASTNode tree has Statement as a child of ASTNode. Statement
> has many children of it's own, one of which is Expression. The problem
> is that what is a statement and an expression is different for each
> language. In Ruby almost everything is an expression, eg. case
> statements, conditional statements, methods etc. This makes building the
> Ruby version of the DLTK AST tree very difficult when Expression is a
> child of Statement.
>
> I suggest a different approach. Remove Statement and Expression
> completely from the tree (ie flatten the tree). Have most AST's subclass
> ASTNode directly except in specific cases such as a list node (although
> another approach would be where ASTNode is either a single node or a
> list of nodes and each node adapts what is returned). This would remove
> the need to perform any casting in the building of each languages AST
> tree. I would add the knowledge of whether a node is an expression or
> not in ASTNode and each language builder would let each node know what
> type of node it is as it's built. This allows the target language to
> define what is an expression, not a predifined inhertiance hierachy.
>
> The AST structure should be defined by the dynamic relationship of nodes
> as they are built not in a static hierarchy tree. Otherwise you get
> structure being defined in two places, the hierachy and the dynamic
> relationship.
>
> Thanks
> Mark
Re: ASTNode - Expression/Statement suggestion [message #2278 is a reply to message #2248] Thu, 26 April 2007 02:34 Go to previous messageGo to next message
Mark Howe is currently offline Mark HoweFriend
Messages: 14
Registered: July 2009
Junior Member
Kool! Thanks Mikhail

Mark


Mikhail Kalugin wrote:
> Hi, Mark.
>
> We have discussed in details this problem inside our team today. Frankly
> speaking we had already thought about solution some months ago, but
> somehow it has left unsolved till this moment.
>
> Anyway, we are fully agree with a suggested solution. We decided to
> implement it in practice, and in the nearest future, I think, problem
> will be solved as you suggested.
>
> Thanks a lot for your help.
>
> --
> Mikhail Kalugin
>
> Mark Howe wrote:
>> I spent the last few days trying to figure out why the SearchEngine
>> doesn't return certain specific types (for instance
>> ActiveRecord::Migration). I realized that if I browse to a type in the
>> ScriptExplorer and open one of the types that the SeachEngine doesn't
>> return there is no structure. Then I realized that JRubySourceParser
>> was eating exceptions (intentionally), so to aid in debugging I
>> allowed the exceptions. Seems like all the problems occurred where an
>> ast was expecting a child AST to be an expression but it was actually
>> a Statement of some kind (like a RubyCaseStatement or an IfStatement).
>>
>> I added some patches to RubyASTBuildVisitor which gets around these
>> problems but it points out a basic problem with the structure of the
>> tree. The ASTNode tree has Statement as a child of ASTNode. Statement
>> has many children of it's own, one of which is Expression. The problem
>> is that what is a statement and an expression is different for each
>> language. In Ruby almost everything is an expression, eg. case
>> statements, conditional statements, methods etc. This makes building
>> the Ruby version of the DLTK AST tree very difficult when Expression
>> is a child of Statement.
>>
>> I suggest a different approach. Remove Statement and Expression
>> completely from the tree (ie flatten the tree). Have most AST's
>> subclass ASTNode directly except in specific cases such as a list node
>> (although another approach would be where ASTNode is either a single
>> node or a list of nodes and each node adapts what is returned). This
>> would remove the need to perform any casting in the building of each
>> languages AST tree. I would add the knowledge of whether a node is an
>> expression or
>> not in ASTNode and each language builder would let each node know what
>> type of node it is as it's built. This allows the target language to
>> define what is an expression, not a predifined inhertiance hierachy.
>>
>> The AST structure should be defined by the dynamic relationship of
>> nodes as they are built not in a static hierarchy tree. Otherwise you
>> get structure being defined in two places, the hierachy and the
>> dynamic relationship.
>>
>> Thanks
>> Mark
Re: ASTNode - Expression/Statement suggestion [message #2396 is a reply to message #2278] Fri, 04 May 2007 11:10 Go to previous message
Eclipse UserFriend
Originally posted by: mikhail.kalugin.xored.com

Hi,

We have done refactoring and now ruby's AST is flat(ruby ast nodes
directly subclasses ASTNode or ASTListNode). You can update from cvs and
see the results.

Statement and Expression classes has been left in DLTK Core to save
code for Tcl and Python, where they are still used, but Core itself
and Ruby should not use them.


Mark Howe wrote:
> Kool! Thanks Mikhail
>
> Mark
>
>
> Mikhail Kalugin wrote:
>> Hi, Mark.
>>
>> We have discussed in details this problem inside our team today.
>> Frankly speaking we had already thought about solution some months
>> ago, but somehow it has left unsolved till this moment.
>>
>> Anyway, we are fully agree with a suggested solution. We decided to
>> implement it in practice, and in the nearest future, I think, problem
>> will be solved as you suggested.
>>
>> Thanks a lot for your help.
>>
>> --
>> Mikhail Kalugin
>>
>> Mark Howe wrote:
>>> I spent the last few days trying to figure out why the SearchEngine
>>> doesn't return certain specific types (for instance
>>> ActiveRecord::Migration). I realized that if I browse to a type in
>>> the ScriptExplorer and open one of the types that the SeachEngine
>>> doesn't return there is no structure. Then I realized that
>>> JRubySourceParser was eating exceptions (intentionally), so to aid in
>>> debugging I allowed the exceptions. Seems like all the problems
>>> occurred where an ast was expecting a child AST to be an expression
>>> but it was actually a Statement of some kind (like a
>>> RubyCaseStatement or an IfStatement).
>>>
>>> I added some patches to RubyASTBuildVisitor which gets around these
>>> problems but it points out a basic problem with the structure of the
>>> tree. The ASTNode tree has Statement as a child of ASTNode. Statement
>>> has many children of it's own, one of which is Expression. The
>>> problem is that what is a statement and an expression is different
>>> for each language. In Ruby almost everything is an expression, eg.
>>> case statements, conditional statements, methods etc. This makes
>>> building the Ruby version of the DLTK AST tree very difficult when
>>> Expression is a child of Statement.
>>>
>>> I suggest a different approach. Remove Statement and Expression
>>> completely from the tree (ie flatten the tree). Have most AST's
>>> subclass ASTNode directly except in specific cases such as a list
>>> node (although another approach would be where ASTNode is either a
>>> single node or a list of nodes and each node adapts what is
>>> returned). This would remove the need to perform any casting in the
>>> building of each languages AST tree. I would add the knowledge of
>>> whether a node is an expression or
>>> not in ASTNode and each language builder would let each node know
>>> what type of node it is as it's built. This allows the target
>>> language to define what is an expression, not a predifined
>>> inhertiance hierachy.
>>>
>>> The AST structure should be defined by the dynamic relationship of
>>> nodes as they are built not in a static hierarchy tree. Otherwise you
>>> get structure being defined in two places, the hierachy and the
>>> dynamic relationship.
>>>
>>> Thanks
>>> Mark
Previous Topic:Apologize, posted int he wrong place
Next Topic:Issues on "A guide to building a DLTK-based language IDE"
Goto Forum:
  


Current Time: Thu Apr 25 15:35:36 GMT 2024

Powered by FUDForum. Page generated in 0.03246 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top