Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [handly-dev] Fw: Re[3]: working copy

Hi Doug, it's nice to get to know you.

Well, first let me say that I don't have more than some foggy ideas and thoughts.

IMHO, the most useful feature of Eclipse (being based on OSGi) is the modularity. It allows to write small, compact libraries that can be understood, reviewed and tested in isolation. In practice, however, many libraries have grown to larger bodies of code that cover multiple areas and assumptions have infiltrated the APIs. When adding the need to provide backwards compatibility, it only gets worse with time.

For example, I claim that the model part of the language support for most languages is copied and adapted from the JDT. However, that code has many Java-specific details that aren't relevant for all languages, making it difficult to understand what is really needed. Also, copying and adapting the code makes it difficult to import any improvements JDT make to their code. So extracting the generic parts, like Handly does, is a big win. 

Projects like DLTK still mix up all the provided functionality in an all-or-nothing package. If one's requirements don't match DLTK's assumptions, then it's not possible to use just parts of DLTK. Similarly, Xtext is exceptional if one can match their assumptions.

So, to return to the topic at hand, I feel that there are more areas that could be separated from the bulk and exposed as generic functionality, areas that otherwise every implementer needs to reinvent (or copy-and-paste from another project). They can't be as standalone as Handly, because some have to depend on a model API.

There is a lot of jface stuff that mixes core and UI: the core part should be possible to extract and let each language implement it, while the UI would be generic. Examples include: content assist, text hover, content formatter, quick assist, reconciler, damager/repairer, projection. At the very least, the non-ui parts should be moved from o.e.jface.text to o.e.text (after fixing the APIs), but it may be useful to have separate bundles. Some of the existing APIs include references to UI details (like ICompletionProposal returning an Image), so these should use some kind of generic token instead. 

Then with these generic (as in: language independent) APIs and a generic model, one could implement a "language engine" that would connect the dots at the next level. It could for example include a database over the language constructs (like Xtext's index) with the associated search mechanism (that would power not only searching for references, but also code completion, etc). But people should be able to set up their own, if they need to, while still using the same basic APIs.

If all works well, it should be possible to have completely generic editors and basic views (i.e. what Xtext generates), with some default implementations that do nothing and let language implementers replace those one by one with relevant ones. 

I know it is difficult to do this in a language-agnostic way and without imposing a certain structure, but today each language implementer needs to reinvent too many wheels, which raises the difficulty of implementing new languages. IntelliJ IDEA has better (but with room for improvements) APIs (I guess it's the second mover's advantage, they learned from Eclipse's first steps and didn't have to be backwards compatible).

Okay, this has become quite a long rant :-) I hope it makes sense.


On Mon, Sep 29, 2014 at 8:09 PM, Vladimir Piskarev <pisv@xxxxx> wrote:
Thanks for your input Doug. It's really great to hear you on this list!

>From my experience with Xtext, I would say that a real C-like preprocessor (with macros) is
an insurmountable barrier. Regarding large files -- it is doable, at least with recent Xtext versions.
(I tried it on source files of up to 70K LOC) Depending on the point of view, the strong EMF dependency
of Xtext may be considered a strength or weakness.

If understand correctly, Vlad is talking about an Eclipse project that would provide a set of common
bricks for IDE rather than a complete framework. Handly has been designed along these lines, but
at the moment supplies only a (small) part of it related to code models. Perhaps Vlad would
elaborate on his points.

I also agree with Vlad regarding DLTK. In fact, LDT (Lua Development Tools) is considering
replacing "a terrible dependency on DLTK" with Handly [1].



> Hey gang, sorry I haven?t been following the discussion closely. But this is a topic area I have a lot of interest in.
> First, for those who don?t know me, I?m a long time committer and lead on the CDT project. I was involved in the creation of CDT?s C and C++ parsers that were created over 10 years ago now. I?ve long followed the progress of ANTLR as a parser generator. The early versions had no chance of parsing C++ and dealing with the preprocessor was a huge problem. In the end, we had to hand code everything and it took us many years to get it right.
> But I think with the new ANTLR version 4, things can be a lot easier. It?s much more forgiving on ambiguities and has some interesting lexer streaming that could help with the preprocessor. Once I get a chance I?m going to try to do Qt?s QML with it and see where that leads.
> But I also like what Xtext does for you. It generates so much of the underlying editor and source navigation code for you. But as you?ve noted, they use their own parser generator and it?s pretty weak. I haven?t followed them closely enough, but my hope is that they one day support ANTLR 4 as an input grammar. Or maybe we need a new project like Xtext for that. I?m not sure what the best way to go, but I?ve found them open in the past to at least listen to my crazy ideas :). It would be a good place to start.
> Doug.
> > Hi!
> >
> >
> > It would be great to have another pair of eyes looking at my code, since almost a year now I am working on it all by myself. I'm taking the conversation off-list, as it might not be of general interest, but feel free to post back to the list if you think it is relevant..
> >
> >
> > There are however some details that affect the way erlide and handly will benefit each other. The biggest problem with this project has always been that it's difficult to find people that know both Java+Eclipse and Erlang. I try to find ways to address this issue and I had two main alternatives:
> > - minimize the Java-related code by using Xtext
> > - minimize the Java-related code by implementing the core functionality in Erlang
> >
> >
> > I gave Xtext a try for a couple of years ago. It went well, but there were two big issues: Erlang has a C-like preprocessor with macros that may make the code look malformed and the Xtext parser choked on that; and there were performance issues for large files. So I was forced to abandon this track...
> >
> >
> > There is already a lot of functionality that is implemented in Erlang, so the other option might work better. The idea is to have the model and all the IDE services implemented in Erlang, which would also let them be used by other IDEs than Eclipse. So on the Eclipse side there should only be a thin layer that delegates to the Erlang implementation.
> >
> >
> > At the moment I'm trying to separate the functionality that will end up being reimplemented in Erlang, so that the work can be done without affecting other parts. The code model is only one part of it, but an important one. I think that the main problem will be how to keep the model synchronized between Java and Erlang -- change notifications would have to flow back and forth quite a lot. Here I think that a well thought-out independent framework like Handly would help a lot, because it extracts the generic parts that can be reviewed and tested separately.
> >
> >
> > Like I mentioned before, it would be great if there was such a framework for the rest of the generic language support; Xtext has something but is dependent on its own parser; DLTK has barely any documentation and is not very active either. Actually, I wasn't very hopeful about Handly either, until I read your "Get Handly" tutorials that are just great -- they make all the difference.
> >
> >
> > Well, that was a rather long introduction, let's keep in touch.
> >
> >
> >
> > best regards,
> >
> > Vlad

handly-dev mailing list
To change your delivery options, retrieve your password, or unsubscribe from this list, visit

Back to the top