Re: [ide-dev] Code editors in the Eclipse (IDE)

[Fabio Zadrozny <fabiofz@xxxxxxxxx>]

On Thu, Jun 16, 2016 at 6:15 AM, Sven Efftinge (sven@xxxxxxxxxxx) <sven@xxxxxxxxxxx> wrote:

Hi Tom,

comments inline...

2016-06-15 13:42 GMT+02:00 Tom Schindl <tom.schindl@xxxxxxxxxxxxxxx>:

The first and most important things needed are:
- syntax highlight (lexical!)
- block detection (for curly brace language this means matching
  {},[],() pairs) used for highlighting and code-folding (although I
  don't really see code-folding as an important feature at all)

I like to separate this between 

 - services that can be done generically / based on "simple" configuration and

 - services that need an AST and workspace/project information

 

The first are executed directly in the editor while the latter need requests to some language engine (or server).

For editor features there are more that come to mind like e.g. auto editing and toggle comment.

There are solutions available for that already:
- Angelo has TextMate support
- LiClipse has TextMate support & its custom spec format
- e(fx)clipse has its own format (who misses code folding as of today)

Although I'm not against having TextMate I repeat that the most
important thing for lexical highlighting is speed hence I'm not sure the
TextMate route is the best way.

Could you elaborate on the performance issues you expect or have faced with interpreting the TextMate grammars in Eclipse?

​I can elaborate a bit (explaining how it was done in LiClipse):

In LiClipse/LiClipseText, what I've done is mostly based on the Eclipse way of doing things, which supports the mechanism of partitioning and scanning (so, the document is split into partitions and later changes to a partition don't need to rescan the whole document).

In TextMate, the grammar is done by specifying a bunch of regexps which may be nested and reference other regexps... So, in LiClipse what was done was mostly by adding custom rules which deal with the TextMate format, but with those rules, it ends up having to reparse much more because of the way they are defined. So, this is a less performant because: 1. it relies on how well the author of the grammar did the regexps as the regexp itself may be a major slowdown and 2. much more needs to be reparsed (LiClipse is actually able to turn some cases into partitions, but that's not always possible and depends a lot on the grammar -- which didn't have this in mind when it was done -- on a number of times, it's possible that the whole document is reparsed).

Also, for other actions later on, it's not enough to just capture the top rule and final scanning, you need to capture all the matches which went on to do each particular character so that other actions it defines later on (such as commenting, auto-indent, etc) work properly (again, LiClipse does keep track of that and gives support for templates, auto-indent, etc. and provides a partitions view to help to develop your own grammar).

So, LiClipse is actually able to work in both worlds: Eclipse itself (when you're willing to develop your own grammar) and TextMate for ease of extension for leveraging existing grammars.

Also, it should be noted that TextMate support is not just syntax highlighting... it also gives things as code-folding, templates, code-indent, commenting, outline, etc (currently LiClipse is not 100% compatible for all of those when dealing with TextMate grammars, but it already goes a long way -- and it does support those well in its own grammar) -- and keep in mind that a grammar may actually reference another grammar in some scope (for instance for HTML with embedded _javascript_) 

-- as a note, I think the current LiClipseText implementation currently already does quite a bit more than the other implementations it's being compared to (although they may catch up later) -- and it's already deployed as a plugin which can already be installed in any Eclipse installation (I know there's a push to put things in Eclipse itself, but personally, I think it's fine living as a separate plugin -- the code is all EPL too, so, it could be even installed in eclipse packages -- I know there are many external libraries living in Eclipse orbit, so, that may be a reasonable place to deploy it too if it really needs to be deployed with Eclipse).

Related to the theming, LiClipseText ends up working with preferences and only works with the theming when installed along with LiClipse (the colors can be customized by an extension point for when theming is available and falls back to preferences in LiClipseText).

Currently LiClipse does have error reporting for some languages (such as XML, HTML and _javascript_), but this currently needs to be done through Eclipse extension points (supporting a common protocol is probably an interesting thing going further on).

Cheers,

Fabio

​