Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Dynamic Languages Toolkit (DLTK) » How about incremental parser!(Has DLTK support for incremental parser?)
How about incremental parser! [message #1091729] Wed, 21 August 2013 22:44 Go to next message
Rodrigo Antonio is currently offline Rodrigo AntonioFriend
Messages: 17
Registered: June 2010
Location: São Paulo - Brasil
Junior Member
Hi,

I Finally finish my editor. Thanks a lot for help.

But I have a little performace problem.

In my editor, every time I type a key, my parser is called(AbstractSourceParser) and I call ANTLR Parser. That's ok, but, there are many sources with 20 thousand lines of code.

I know that has a lot of lines in source isn't a good ideia, but we have many legacy sources.

Well,

I would like to do incremental parser, like JDT.

Has DLTK support for that?

Regards,
Rodrigo
Re: How about incremental parser! [message #1092717 is a reply to message #1091729] Fri, 23 August 2013 05:55 Go to previous messageGo to next message
Alex Panchenko is currently offline Alex PanchenkoFriend
Messages: 342
Registered: July 2009
Senior Member
Hi Rodrigo,

The parser from the editor is usually called with a small delay (0.5
second) to refresh the module structure (e.g. Outline), this operation
is called "reconcile".

As I understand, when you edit a file, only that single file should be
re-parsed.

Actually JDT has no incremental parser (or probably you mean something
different). It uses hand-written parser which is way faster than ANTLR
generated and can work in 2 passes - first extract classes and
functions, then (when needed) go deeper and parse each statement and
build full AST. For example, when refreshing java file structure
(Outline), only the 1st step is enough.

If your language grammar allows such an optimization - you can try
implementing this approach. Otherwise the only way is
profiling/optimizing the current parser/validations, and delaying some
operations if possible.

If files are always big and parser can't be optimized then the only
alternative could be disabling the reconciler at all.

Hope this helps.

Regards,
Alex
Re: How about incremental parser! [message #1095203 is a reply to message #1092717] Mon, 26 August 2013 19:38 Go to previous message
Rodrigo Antonio is currently offline Rodrigo AntonioFriend
Messages: 17
Registered: June 2010
Location: São Paulo - Brasil
Junior Member
Hello Alex,

Thanks again for the reply.

You are correct, I need compile only one source at a time but they are really big in some cases.

Studying this excellent article on IBM:

http://www.ibm.com/developerworks/opensource/tutorials/os-ecl-commplgin1/

I thought I'd do this:

- Determine where the function source code is being edited.
- Catch the beginning and the end of this function.
- Instantiate the ANTLR Parser, passing as input only this function.
- Create a new DLTK MODEL for this Function.
- Remove the old DLTK Model
- Add New DLTK Model

However, to do what I described above, I need to be able to do this:

- Some where to receiving the IRegion when we're typing in source.( I don't get this information in my AbstractSourceParser.
- How can I delete only one IMethod, and add other iMethod?


I'm studing also possible optimizations in ANTLR.

Regards,
Rodrigo
Previous Topic:Ruby code completion on existing files not working?
Next Topic:Atomic Types and global methods
Goto Forum:
  


Current Time: Fri Nov 28 11:32:56 GMT 2014

Powered by FUDForum. Page generated in 0.02976 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software