Eclipse Community Forums: TMF (Xtext) » Very poor UI performance for large files in xtext

Home » Modeling » TMF (Xtext) » Very poor UI performance for large files in xtext(Very poor UI performance for large files in xtext)

Show: Today's Messages :: Show Polls :: Message Navigator

Very poor UI performance for large files in xtext [message #1237981]

Fri, 31 January 2014 04:43

Arshad Adavani

Messages: 163
Registered: July 2013
Location: Bangalore

Senior Member

Hi all,
I have a xtext editor which has a lot of contents.
If I consider LoC then it has around 130000 LoC and UI response is very poor.
UI always goes in NOT RESPONDING state when I try to make some changes in the editor.

Can anyone suggest me some improvements please.

Thanks in advance,
Arshad

Arshad

Report message to a moderator

Re: Very poor UI performance for large files in xtext [message #1237997 is a reply to message #1237981]

Fri, 31 January 2014 05:53

Christian Dietrich

Messages: 14665
Registered: July 2009

Senior Member

Use a Profiler and See where the Performance is lost.
How many Files are that many loc. Which customizations ( scoping etc) did you do ....

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Very poor UI performance for large files in xtext [message #1238007 is a reply to message #1237981]

Fri, 31 January 2014 06:32

Christian Dietrich

Messages: 14665
Registered: July 2009

Senior Member

P.s. if you have all these locs in one file I doubt Xtext will
perform well anyway. But who seariously wants to edit such a huge
file...

--
Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext at itemis dot de

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Very poor UI performance for large files in xtext [message #1238045 is a reply to message #1238007]

Fri, 31 January 2014 08:50

Arshad Adavani

Messages: 163
Registered: July 2013
Location: Bangalore

Senior Member

Hi Christian,
I have such a big input file that it will be more than 100000 LoC and its the real data.
I have done customizations such as syntax highlighting, outlining, scoping and quickfix etc.
I am not getting how to deal with this now.

Thanks in advance Smile

Arshad

Report message to a moderator

Re: Very poor UI performance for large files in xtext [message #1238059 is a reply to message #1238045]

Fri, 31 January 2014 09:37

Christian Dietrich

Messages: 14665
Registered: July 2009

Senior Member

Hi why do use xtext at all?

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Very poor UI performance for large files in xtext [message #1238094 is a reply to message #1238045]

Fri, 31 January 2014 11:21

Ed Willink

Messages: 7655
Registered: July 2009

Senior Member

Hi

Indeed; why use Xtext?

Some quick guesstimates.

100000LoC, 40 characters per line => 4 MB not a real problem for a
direct text tool.

10 tokens per line => 1 M Token for a parsed token representation
(with 10 4 byte fields per token => 40 MB) big but not impossible.
Probably the same again for a semantic representation.
And the same again for an Outline View.
And ...

The above are what an ideal tool might involve, and perhaps Xtext is not
yet ideal.

Conclusion: if you want mega-text, modularize or use a tool (emacs?)
that works on the text directly.

Regards

Ed Willink

On 31/01/2014 08:50, Arshad Adavani wrote:
> Hi Christian,
> I have such a big input file that it will be more than 100000 LoC and
> its the real data.
> I have done customizations such as syntax highlighting, outlining,
> scoping and quickfix etc.
> I am not getting how to deal with this now.
>
>
> Thanks in advance :)

Report message to a moderator

Re: Very poor UI performance for large files in xtext [message #1239431 is a reply to message #1238045]

Tue, 04 February 2014 07:39

Sven Efftinge

Messages: 1823
Registered: July 2009

Senior Member

Hi Arshad,

there are many possibilities to improve the performance of your editor.
The defaults are more focussed on functionality than on scaling to such
big files. It really depends on your language, the customizations you
did and the compromises you would be ok with ;-).

Note, that we (itemis) do this kind of optimizations a lot. So if you
can afford to spend some money on that, we will be pleased to do the job
for you.

If you want to do that on your own, the profiler is your friend (As
Christian already pointed out).

Cheers,
Sven

Am 31/01/14 09:50, schrieb Arshad Adavani:
> Hi Christian,
> I have such a big input file that it will be more than 100000 LoC and
> its the real data.
> I have done customizations such as syntax highlighting, outlining,
> scoping and quickfix etc.
> I am not getting how to deal with this now.
>
>
> Thanks in advance :)

--
Need professional support for Xtext or other Eclipse Modeling technologies?
Go to: http://xtext.itemis.com
Twitter : @svenefftinge
Blog : http://blog.efftinge.de

Report message to a moderator

Re: Very poor UI performance for large files in xtext [message #1240915 is a reply to message #1239431]

Fri, 07 February 2014 08:23

Joost van Pinxten

Messages: 51
Registered: November 2012

Member

I have noticed similar behavior. With just the parser, linker and validator generated from source, I get a non-responsive editor. In standalone mode, I've profiled the different parts:

Reading a text file into a stringbuffer: 17 MB in 170ms (~100MB/s)
Parsing the resulting text with the generated parser: 17 MB in ~10s (~1.7MB/s)
Calling validate (with the empty generated validator) on this model: 948M elements in 3600+ s.

Profiling leads to a lot of calls to .eget, where the proxies in the references need to be resolved. So even though I've removed all @Check annotations, I am bound by this lower limit in performance. Scoping does not show up significantly at all in the profiler.

It would be helpful to at least have some pointers to which components are not built for scalable design. I do understand that not disclosing detailed information is in the best interest for Itemis as a company...

For small projects, I reckon it's already an awesome project, and quite close to being 'ideal' (as hinted by Ed Willink).

Report message to a moderator

Re: Very poor UI performance for large files in xtext [message #1242099 is a reply to message #1237981]

Sun, 09 February 2014 02:05

Henrik Lindberg

Messages: 2509
Registered: July 2009

Senior Member

On 2014-31-01 5:43, Arshad Adavani wrote:
> Hi all,
> I have a xtext editor which has a lot of contents.
> If I consider LoC then it has around 130000 LoC and UI response is very
> poor.
> UI always goes in NOT RESPONDING state when I try to make some changes
> in the editor.
>
> Can anyone suggest me some improvements please.
>

I had similar problems in Geppetto (for the Puppet language). In my case
there were too many rules and tokens which created huge ASTs. I solved
this by moving complexities from the grammar to an external lexer.

The second problem I experienced was the speed of linking and I ended up
implementing my own using a different way to index data. My
implementation is specific to Geppetto and I basically do not use any
model references and instead just assign data values (that I custom
link). I could have used real references I guess, but I did not know how
to make it perform.

I would look at the generated tree first. If you have lots of
"unnecessary" tokens (and deep rule hierarchies) it is a win-win to
solve this in the lexer as it will both lex faster, simplify the rules,
and generate smaller trees (which gives a speed improvement almost
everywhere).

Or, as Sven said - get some help from Itemis, at least to figure out
what to improve.

Regards
- henrik

Report message to a moderator

Re: Very poor UI performance for large files in xtext [message #1243028 is a reply to message #1240915]

Mon, 10 February 2014 12:10

Sven Efftinge

Messages: 1823
Registered: July 2009

Senior Member

Am 07/02/14 09:23, schrieb Joost van Pinxten:
> It would be helpful to at least have some pointers to which components
> are not built for scalable design. I do understand that not disclosing
> detailed information is in the best interest for Itemis as a company...

There's definitely nothing built to perform poor such that we have more
business.
It's only that we have some experience with tuning Xtext applications.
But it is really the best to profile your specific use case and see
where you can optimize or simply avoid functionality.

In the editor for instance the lexer is triggered a lot and sync with
UI. parsing and validation is done in the background and wouldn't block
the UI unless some other action needs to access the readily parsed model.

Sorry, but there is no easy answer nor some special knowledge we keep
secret. It's merely experience that makes our support services valuable.

Sven

--
Need professional support for Xtext or other Eclipse Modeling technologies?
Go to: http://xtext.itemis.com
Twitter : @svenefftinge
Blog : http://blog.efftinge.de

Report message to a moderator

Re: Very poor UI performance for large files in xtext [message #1243174 is a reply to message #1243028]

Mon, 10 February 2014 16:18

Arshad Adavani

Messages: 163
Registered: July 2013
Location: Bangalore

Senior Member

Hi all,
First of all thank you all for sharing this info with me.
But what makes me happy is that MY EDITOR PERFORMANCE IS BACK.

As Christian and others rightly said, performance will not be good for such a big file which has around 130000 LoC with all other customizations done (i.e. scoping, outlining, syntax highlighting etc).

But now I have divided this data into multiple DSLs which have same namespace and this works out as expected for me!
This data is divided based on some requirements and everyone's happy with it. Smile

But just I want to ask it one more time, for a file which has around 130000 LoC, do you people think any further optimizations can be done except dividing it into multiple DSLs ??

Thank you all Smile

Arshad

Report message to a moderator

Re: Very poor UI performance for large files in xtext [message #1243230 is a reply to message #1243028]

Mon, 10 February 2014 17:57

Joost van Pinxten

Messages: 51
Registered: November 2012

Member

Sven Efftinge wrote on Mon, 10 February 2014 07:10

There's definitely nothing built to perform poor such that we have more
business.
It's only that we have some experience with tuning Xtext applications.
But it is really the best to profile your specific use case and see
where you can optimize or simply avoid functionality.

I must appologize, as I see that I chose my words poorly; I did not intend to imply that they have been deliberately built to perform poorly. One thing that I've seen around the forums and in related projects quite a few times now, is that resolving lazy references (which is implied when validating) seems to take an exponential amount of time.

Profiling with VisualVM has caused me a lot of frustrations, as it seems the thread (launched from Eclipse JUnit) that actually executes the code, is not profiled at all (which could be related to me choosing such 'harsh' words here in the first place). But that's a story for a different post Wink

Quote:

In the editor for instance the lexer is triggered a lot and sync with
UI. parsing and validation is done in the background and wouldn't block
the UI unless some other action needs to access the readily parsed model.

Sorry, but there is no easy answer nor some special knowledge we keep
secret. It's merely experience that makes our support services valuable.

I have propagated the message that smaller files should remedy this problem to some extent. This also means that the DSL that I am developing should be structured in a different way...

Report message to a moderator

Previous Topic:	Retrieving / reverse engineering grammar file from existing JavaCC generated parser
Next Topic:	different instances of editor were given

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Sat Apr 20 00:43:55 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter