Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Very poor UI performance for large files in xtext(Very poor UI performance for large files in xtext)
icon5.gif  Very poor UI performance for large files in xtext [message #1237981] Fri, 31 January 2014 04:43 Go to next message
Arshad Adavani is currently offline Arshad AdavaniFriend
Messages: 163
Registered: July 2013
Location: Bangalore
Senior Member
Hi all,
I have a xtext editor which has a lot of contents.
If I consider LoC then it has around 130000 LoC and UI response is very poor.
UI always goes in NOT RESPONDING state when I try to make some changes in the editor.

Can anyone suggest me some improvements please.


Thanks in advance,
Arshad


Arshad
Re: Very poor UI performance for large files in xtext [message #1237997 is a reply to message #1237981] Fri, 31 January 2014 05:53 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13655
Registered: July 2009
Senior Member
Use a Profiler and See where the Performance is lost.
How many Files are that many loc. Which customizations ( scoping etc) did you do ....


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Very poor UI performance for large files in xtext [message #1238007 is a reply to message #1237981] Fri, 31 January 2014 06:32 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13655
Registered: July 2009
Senior Member
P.s. if you have all these locs in one file I doubt Xtext will
perform well anyway. But who seariously wants to edit such a huge
file...

--
Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext at itemis dot de


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
icon5.gif  Re: Very poor UI performance for large files in xtext [message #1238045 is a reply to message #1238007] Fri, 31 January 2014 08:50 Go to previous messageGo to next message
Arshad Adavani is currently offline Arshad AdavaniFriend
Messages: 163
Registered: July 2013
Location: Bangalore
Senior Member
Hi Christian,
I have such a big input file that it will be more than 100000 LoC and its the real data.
I have done customizations such as syntax highlighting, outlining, scoping and quickfix etc.
I am not getting how to deal with this now.


Thanks in advance Smile


Arshad
Re: Very poor UI performance for large files in xtext [message #1238059 is a reply to message #1238045] Fri, 31 January 2014 09:37 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13655
Registered: July 2009
Senior Member
Hi why do use xtext at all?

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Very poor UI performance for large files in xtext [message #1238094 is a reply to message #1238045] Fri, 31 January 2014 11:21 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6924
Registered: July 2009
Senior Member
Hi

Indeed; why use Xtext?

Some quick guesstimates.

100000LoC, 40 characters per line => 4 MB not a real problem for a
direct text tool.

10 tokens per line => 1 M Token for a parsed token representation
(with 10 4 byte fields per token => 40 MB) big but not impossible.
Probably the same again for a semantic representation.
And the same again for an Outline View.
And ...

The above are what an ideal tool might involve, and perhaps Xtext is not
yet ideal.

Conclusion: if you want mega-text, modularize or use a tool (emacs?)
that works on the text directly.

Regards

Ed Willink



On 31/01/2014 08:50, Arshad Adavani wrote:
> Hi Christian,
> I have such a big input file that it will be more than 100000 LoC and
> its the real data.
> I have done customizations such as syntax highlighting, outlining,
> scoping and quickfix etc.
> I am not getting how to deal with this now.
>
>
> Thanks in advance :)
Re: Very poor UI performance for large files in xtext [message #1239431 is a reply to message #1238045] Tue, 04 February 2014 07:39 Go to previous messageGo to next message
Sven Efftinge is currently offline Sven EfftingeFriend
Messages: 1823
Registered: July 2009
Senior Member
Hi Arshad,

there are many possibilities to improve the performance of your editor.
The defaults are more focussed on functionality than on scaling to such
big files. It really depends on your language, the customizations you
did and the compromises you would be ok with ;-).

Note, that we (itemis) do this kind of optimizations a lot. So if you
can afford to spend some money on that, we will be pleased to do the job
for you.

If you want to do that on your own, the profiler is your friend (As
Christian already pointed out).

Cheers,
Sven

Am 31/01/14 09:50, schrieb Arshad Adavani:
> Hi Christian,
> I have such a big input file that it will be more than 100000 LoC and
> its the real data.
> I have done customizations such as syntax highlighting, outlining,
> scoping and quickfix etc.
> I am not getting how to deal with this now.
>
>
> Thanks in advance :)


--
Need professional support for Xtext or other Eclipse Modeling technologies?
Go to: http://xtext.itemis.com
Twitter : @svenefftinge
Blog : http://blog.efftinge.de
Re: Very poor UI performance for large files in xtext [message #1240915 is a reply to message #1239431] Fri, 07 February 2014 08:23 Go to previous messageGo to next message
Joost van Pinxten is currently offline Joost van PinxtenFriend
Messages: 51
Registered: November 2012
Member
I have noticed similar behavior. With just the parser, linker and validator generated from source, I get a non-responsive editor. In standalone mode, I've profiled the different parts:

Reading a text file into a stringbuffer: 17 MB in 170ms (~100MB/s)
Parsing the resulting text with the generated parser: 17 MB in ~10s (~1.7MB/s)
Calling validate (with the empty generated validator) on this model: 948M elements in 3600+ s.

Profiling leads to a lot of calls to .eget, where the proxies in the references need to be resolved. So even though I've removed all @Check annotations, I am bound by this lower limit in performance. Scoping does not show up significantly at all in the profiler.

It would be helpful to at least have some pointers to which components are not built for scalable design. I do understand that not disclosing detailed information is in the best interest for Itemis as a company...

For small projects, I reckon it's already an awesome project, and quite close to being 'ideal' (as hinted by Ed Willink).
Re: Very poor UI performance for large files in xtext [message #1242099 is a reply to message #1237981] Sun, 09 February 2014 02:05 Go to previous messageGo to next message
Henrik Lindberg is currently offline Henrik LindbergFriend
Messages: 2509
Registered: July 2009
Senior Member
On 2014-31-01 5:43, Arshad Adavani wrote:
> Hi all,
> I have a xtext editor which has a lot of contents.
> If I consider LoC then it has around 130000 LoC and UI response is very
> poor.
> UI always goes in NOT RESPONDING state when I try to make some changes
> in the editor.
>
> Can anyone suggest me some improvements please.
>

I had similar problems in Geppetto (for the Puppet language). In my case
there were too many rules and tokens which created huge ASTs. I solved
this by moving complexities from the grammar to an external lexer.

The second problem I experienced was the speed of linking and I ended up
implementing my own using a different way to index data. My
implementation is specific to Geppetto and I basically do not use any
model references and instead just assign data values (that I custom
link). I could have used real references I guess, but I did not know how
to make it perform.

I would look at the generated tree first. If you have lots of
"unnecessary" tokens (and deep rule hierarchies) it is a win-win to
solve this in the lexer as it will both lex faster, simplify the rules,
and generate smaller trees (which gives a speed improvement almost
everywhere).

Or, as Sven said - get some help from Itemis, at least to figure out
what to improve.

Regards
- henrik
Re: Very poor UI performance for large files in xtext [message #1243028 is a reply to message #1240915] Mon, 10 February 2014 12:10 Go to previous messageGo to next message
Sven Efftinge is currently offline Sven EfftingeFriend
Messages: 1823
Registered: July 2009
Senior Member
Am 07/02/14 09:23, schrieb Joost van Pinxten:
> It would be helpful to at least have some pointers to which components
> are not built for scalable design. I do understand that not disclosing
> detailed information is in the best interest for Itemis as a company...

There's definitely nothing built to perform poor such that we have more
business.
It's only that we have some experience with tuning Xtext applications.
But it is really the best to profile your specific use case and see
where you can optimize or simply avoid functionality.

In the editor for instance the lexer is triggered a lot and sync with
UI. parsing and validation is done in the background and wouldn't block
the UI unless some other action needs to access the readily parsed model.

Sorry, but there is no easy answer nor some special knowledge we keep
secret. It's merely experience that makes our support services valuable.

Sven


--
Need professional support for Xtext or other Eclipse Modeling technologies?
Go to: http://xtext.itemis.com
Twitter : @svenefftinge
Blog : http://blog.efftinge.de
icon14.gif  Re: Very poor UI performance for large files in xtext [message #1243174 is a reply to message #1243028] Mon, 10 February 2014 16:18 Go to previous messageGo to next message
Arshad Adavani is currently offline Arshad AdavaniFriend
Messages: 163
Registered: July 2013
Location: Bangalore
Senior Member
Hi all,
First of all thank you all for sharing this info with me.
But what makes me happy is that MY EDITOR PERFORMANCE IS BACK.

As Christian and others rightly said, performance will not be good for such a big file which has around 130000 LoC with all other customizations done (i.e. scoping, outlining, syntax highlighting etc).

But now I have divided this data into multiple DSLs which have same namespace and this works out as expected for me!
This data is divided based on some requirements and everyone's happy with it. Smile


But just I want to ask it one more time, for a file which has around 130000 LoC, do you people think any further optimizations can be done except dividing it into multiple DSLs ??


Thank you all Smile


Arshad
Re: Very poor UI performance for large files in xtext [message #1243230 is a reply to message #1243028] Mon, 10 February 2014 17:57 Go to previous message
Joost van Pinxten is currently offline Joost van PinxtenFriend
Messages: 51
Registered: November 2012
Member
Sven Efftinge wrote on Mon, 10 February 2014 07:10

There's definitely nothing built to perform poor such that we have more
business.
It's only that we have some experience with tuning Xtext applications.
But it is really the best to profile your specific use case and see
where you can optimize or simply avoid functionality.


I must appologize, as I see that I chose my words poorly; I did not intend to imply that they have been deliberately built to perform poorly. One thing that I've seen around the forums and in related projects quite a few times now, is that resolving lazy references (which is implied when validating) seems to take an exponential amount of time.

Profiling with VisualVM has caused me a lot of frustrations, as it seems the thread (launched from Eclipse JUnit) that actually executes the code, is not profiled at all (which could be related to me choosing such 'harsh' words here in the first place). But that's a story for a different post Wink

Quote:

In the editor for instance the lexer is triggered a lot and sync with
UI. parsing and validation is done in the background and wouldn't block
the UI unless some other action needs to access the readily parsed model.

Sorry, but there is no easy answer nor some special knowledge we keep
secret. It's merely experience that makes our support services valuable.


I have propagated the message that smaller files should remedy this problem to some extent. This also means that the DSL that I am developing should be structured in a different way...
Previous Topic:Retrieving / reverse engineering grammar file from existing JavaCC generated parser
Next Topic:different instances of editor were given
Goto Forum:
  


Current Time: Tue Nov 24 04:54:40 GMT 2020

Powered by FUDForum. Page generated in 0.02041 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top