Eclipse Community Forums: IMP » ide for a language embedded in java

Help

Home

Home » Archived » IMP » ide for a language embedded in java(is IMP the best choice?)

Show: Today's Messages :: Show Polls :: Message Navigator

ide for a language embedded in java [message #505310]

Tue, 29 December 2009 13:38

giovanni

Messages: 18
Registered: November 2009

Junior Member

Hi all, I am a newcomer looking for some sapient advice.
I've read a lot about IMP, DLTK, XText and JDT,
but now I need some suggestion to get the right way.
We are working on a language embedded in Java, thinks something like:

class Foo{
public int myMethod(){
int num=17;
return <% my syntax that use num to do something %>;
}
}

From this code, a regular java file will be generated, properly replacing the <% %> blocks in some way.
I'd like to provide an IDE for that.
Being a superset of java, would be wonderful to have full support of JDT, plus some features for the custom part.
I'm not sure which is the project that better fits my problem and save me a lot of work.
For example I don't want to write from scratch a java grammar and implement a whole IDE for it, but possibly extending an existent one (seems IMP provides an example of Java IDE).
In your opinion, which is the best choice or way to follow?
Any contribution is really appreciate.
Thanks

Report message to a moderator

Re: ide for a language embedded in java [message #507808 is a reply to message #505310]

Thu, 14 January 2010 17:51

Stephan Herrmann

Messages: 1853
Registered: July 2009

Senior Member

Hi Giovanni,

> Hi all, I am a newcomer looking for some sapient advice.
> I've read a lot about IMP, DLTK, XText and JDT,
> but now I need some suggestion to get the right way.

That's quite a spectrum ;-)
I'm not an IMP expert, but I have experience in adapting the JDT.

> We are working on a language embedded in Java, thinks something like:
>
> class Foo{
> public int myMethod(){
> int num=17;
> return <% my syntax that use num to do something %>;
> }
> }
>
> From this code, a regular java file will be generated, properly replacing
> the <% %> blocks in some way.

Is it an option for you to (manually?) translate your syntax into
Java AST rather than Java source code? If so, I could envision the
following strategy:

- adapt the JDT Scanner so that it recognizes the <% token.
- hook your own translator that will feed synthesized AST
into the JDT Parser
(some more magic on the parser may be required here).
- let the JDT do all the rest

In this strategy it would be crucial to create AST nodes with the
original source positions. This could make features like F3-navigation,
semantic highlighting etc. work out-of-the box.

If this sounds reasonable to you, I could guide you to using
OT/Equinox ( http://trac.objectteams.org/ot/wiki/OtEquinox )
as the technology for hooking your translator into the JDT.
With OT/Equinox you can also adapt any UI features of the JDT to
better handle your syntax. In order to get an idea of what we have
done with this strategy you may check this presentation:
The Object Teams Development Tooling:
A high fidelity extension of the JDT
Long Talk at EclipseCon 2008, Santa Clara, USA.
http://www.eclipsecon.org/2008/index.php?page=sub/&id=35 2

This approach dosn't use any cool code generators but supports
maximum re-use of the JDT.

BTW: Object Teams (and thus OT/Equinox) are currently moving to Eclipse.

Let me know if you'd like more details in this direction.

cheers,
Stephan

Report message to a moderator

Re: ide for a language embedded in java [message #508559 is a reply to message #507808]

Tue, 19 January 2010 12:52

giovanni

Messages: 18
Registered: November 2009

Junior Member

Hi Stephan,
thanks for your message.
After my first post, we went on looking for the best or at least a good approach, finding that java code manipulation by
AST is really a powerful tool (quick fix and refactoring are excellent examples).
We still are in a preliminary analysis phase so any help is appreciate.
Because we'd like to have unrestricted Java development using JDT, and because we didn't know how to adapt the JDT,
the current idea we are investigating considers comments overloading. For example

List<Person> people;
/*#
* my syntax that uses people to do something
*/

The special char # after /* marks a custom comment. Using AST rewriting we "should" be able to replace it with the proper (AST) Java code.
Of course this approach is quite raw and oversimplified and could lead to troubles that now I'm not aware of, having bit
experience of Eclipse development. (anyway I'd appreciate to know what do you think about that).
A pro is that is framework independent, because one can use this translator outside Eclipse.

Said that, well, your work on OTDT is absolutely amazing!! It truly seems very suitable for our purposes:
we could define a complex syntax in special tags (<% or similar), having code completion and all the others eclipse features.
Such a tool would be wonderful to me.
Anyway there are some points not very clear to me:

1) In "(manually?) translate your syntax into Java AST", what does manually means?

2) In OTDT the user writes a .java file that will be compiled to .class.
In our framework, we envisage users will write a .java (plus customs) file that will produce either a new .java file
(after translation) or a write back on the original file (as for refactoring). Do you think it is possible?

3) Adopting the strategy you suggest, how much is actually difficult to modify JDT scanner and parser? In addition,
when the translation is invoked we need knowledge about the scope context:
in the example above we must know that people is a List parameterized with Person.

4) OtEquinox should be the core technology to achieve that: are there tutorials, code examples etc in order to
getting started and get support?

Thanks again. I'm looking forward to understanding if I can use your technology.
Giovanni

Report message to a moderator

Re: ide for a language embedded in java [message #509033 is a reply to message #508559]

Thu, 21 January 2010 08:46

Stephan Herrmann

Messages: 1853
Registered: July 2009

Senior Member

Giovanni,

Your use case really made me curious *how* easily such thing can be done
with OT/Equinox. I ended up writing a tiny plugin demonstrating who to
hook into the JDT (Scanner & Parser at this point).
I've uploaded it to
http://www.objectteams.org/distrib/demos/embedded-expression -language.zip

In order to spare the IMP forum from further details on OT/Equinox
I will comment on this in a minute at this forum:
http://www.eclipse.org/forums/eclipse.objectteams

Some direct answers below:

> 1) In "(manually?) translate your syntax into Java AST", what does
> manually means?

I give you (char[] source, int start, int end) and from there on,
it's up to you how to parse your sub-language and create Java AST from it.

> 2) In OTDT the user writes a .java file that will be compiled to .class.
> In our framework, we envisage users will write a .java (plus customs) file
> that will produce either a new .java file (after translation) or a write
> back on the original file (as for refactoring). Do you think it is
> possible?

Impossible is nothing. Do you really expect the user to work with the
translated .java file, like editing? Instead of all the resulting hassle
with keeping both versions consistent I'd really favor to do everything
on the fly, just write out the full translated .class, no?
The tool will be much simpler that way.

> 3) Adopting the strategy you suggest, how much is actually difficult to
> modify JDT scanner and parser?

100 lines of code (incl. comments) ;-)

> In addition, when the translation is
> invoked we need knowledge about the scope context: in the example above we
> must know that people is a List parameterized with Person.

If you subclass, e.g., org.eclipse.jdt.internal.compiler.ast.Expression
the compiler will invoke "TypeBinding resolveType(BlockScope scope)"
where the scope contains all the info you need. But that's not during
parsing of course, but later during resolving. One more reason to work
on-the-fly: you have to let the compiler run completely just to get to
the resolving stage. Why then dump .java source and start from the
beginning again, rather than doing just a few more steps and get the
full .class?

> 4) OtEquinox should be the core technology to achieve that: are there
> tutorials, code examples etc in order to getting started and get support?

The first starting point would be
http://trac.objectteams.org/ot/wiki/QuickStart
It contains links into the main sections of the OT/J Language Definition.
But perhaps a description of my little experiment will give the fasted
start.

to be continued at eclipse.objectteams ;-)
Stephan

Report message to a moderator