Re: [cdt-core-dev] Prototype AST2 available in cdt_ast2

[Douglas Schaefer <dschaefe@xxxxxxxxxx>]

cdt-core-dev-admin@xxxxxxxxxxx wrote on 09/28/2004 04:55:02 AM:

> On Mon, 20 Sep 2004 16:23:19 -0400, Douglas Schaefer 
<dschaefe@xxxxxxxxxx> 
> wrote:
> 
> > Hey gang,
> >
> > I've taken a quick stab at a prototype implementation of the new AST. 
> > This work, of course, is being done on the cdt_ast2 branch. I have 
> > created a
> > new JUnit test class AST2Tests in core.tests. The AST interface and
> > factory class are in org.eclipse.cdt.core.parser.ast2 in the parser
> > folder. The AST implementation, including a source element requestor 
to
> > reuse our parser to create the AST, are in
> > org.eclipse.cdt.internal.core.parser.ast2.
> >
> > So far, I am just parsing "int x;", but it lets me get a feel for the
> > architecture and applicability of the AST.
> >
> > Any feedback is appreciated, knowing of course this is a pretty rough 
run
> > at it. Discussion of this feature is being done at:
> > https://bugs.eclipse.org/bugs/show_bug.cgi?id=57816.
> >
> > Cheers,
> > Doug Schaefer, IBM's Eclipse CDT Architect
> > Ottawa (Palladium), Ontario, Canada
> 
> Hi there,
> 
> I'm quite new to this game (that is, to Eclipse), so forgive me if I'm 
> going out the wrong branch. I have

You've reached the right place. Thanks for the feedback!
 
> I read through you paper and the basic design seems quite good, but I?m 
> not clear on some distinctions between difference kind of nodes, e.g.: 
> What is the difference between a variable and an identifier? And what is 
 
> the difference between a reference and an identifier?

An identifier is simply a name in the program text. A declaration 
introduces a name into scope and associates it with a variable, function, 
or type. I am starting to refer to variables, functions, and types as 
'definitions'. A reference is the use by name of a definition somewhere in 
the program, e.g., the use of a variable in an expression, the use of a 
type in a declaration.

> Another thing that I?m more concerned about is the extensibility of the 
> AST. The current design is very focused on languages in the Algol 
family; 
> which is a good starting point but (IMHO) to be practical, the design 
need 
> to be extended to handle a wider domain of languages.
>
> My reasoning goes: The AST is intended to be used for refactoring 
purposes 
> in, at least, C/C++ projects and in addition "easy" extensions to other 
> languages (such as Ada, Pascal, Modula-2, etc.) So, to be useful there 
> have to be methods to handle at least the following language 
constructions:
> - tasks in Ada (which might not be a big problem)
> - macros in C/C++ (I can't see any support for that kind of object)
> - grammar rules in Bison (this is commonly used in C/C++ projects)
> - patterns and actions in Flex (also common in C/C++ projects)
> 
> The last two I find most interesting since Bison and Flex are very 
> different from C and C++ and are commonly used in C/C++ projects. I 
> believe that designing the AST so that it is possible to handle Bison 
and 
> Flex specifications would help creating a AST design that is language 
> independent enough to be able to be used widely.
> 
> So, I'm interested about how to handle Yacc/Bison and Lex/Flex 
> specification, allowing Eclipse to refactor and generate code for these. 
 
> Provided a suitable AST design is developed, I could volunteer to write 
> parsers for both Lex and Yacc specifications (on my personal time).

I've considered this. My objective with the AST is to be able to implement 
core features in as language independent of a manner as possible. For 
example for some new language that we handle, once you build a parser to 
create the AST, you would get some minimal functionality such as content 
assist for free. To do that practically, I am limiting the support for 
C-like languages (not having studied Algol myself I suppose we are 
Algol-like as well).

The plan is to extends the AST for new concepts but relate them back to 
the language independent concepts. For example, Ada tasks could be 
represented as special functions. Since we're not perfectionists, we will 
make exceptions for things we need to represent in the AST to fully 
capture the contents of the source code. C macros would fall into this 
category since we'll need that info to properly implement refactoring.

Can we do bison and flex? That sounds pretty difficult but theorectically 
possible. The most difficult part will be constructing a useful AST out of 
it. Feel free to experiment and let us know how if this is actually 
possible.

Thanks again,
Doug