Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Sirius » Sirius benchmark
Sirius benchmark [message #1774150] Wed, 11 October 2017 03:48 Go to next message
Bo Zhang is currently offline Bo ZhangFriend
Messages: 28
Registered: July 2017
Junior Member
I saw here :http://www.tuicool.com/articles/j2QJRz2.
It said "Sirius performances are in constant improvements, and this version comes with significant enhancements on heap consumption and time execution.".

I imported the odesign project from https://github.com/cbrun/sirius-interpreters-benchmark
and import the 1M elements project from https://bugs.eclipse.org/bugs/attachment.cgi?id=254581.

It work very very slow when we have 1M elements, and I can't open the class diagram.

So I want to know how did you test the performence of sirius? and the hardware information.

I wish for replys.
Re: Sirius benchmark [message #1774163 is a reply to message #1774150] Wed, 11 October 2017 08:40 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6989
Registered: July 2009
Senior Member
Hi

You shouldn't expect too much with 1M graphical elements. Graphical representations have non-trivial overheads.

How big is your VM? You can see from Fig 16 of [1] that even manual code starts to fail at 10M EMF elements in a 4GB VM. A simple EMF activity such as EcoreUtil copy may take nearly 10 seconds on 10M elements on an ordinary PC. Poor code may easily go unacceptably quadratic.

In practice there are no doubt many algorithms in Sirius that can be improved, but until then you would be well advised to consider approaches that reduce the number of model elements. Otherwise get a really big fast PC.

Regards

Ed Willink

[1] http://www.eclipse.org/mmt/qvt/docs/ICMT2017/MicromappingMoC.pdf
Re: Sirius benchmark [message #1774173 is a reply to message #1774163] Wed, 11 October 2017 09:44 Go to previous messageGo to next message
Bo Zhang is currently offline Bo ZhangFriend
Messages: 28
Registered: July 2017
Junior Member
Ed Willink wrote on Wed, 11 October 2017 16:40
Hi

You shouldn't expect too much with 1M graphical elements. Graphical representations have non-trivial overheads.

How big is your VM? You can see from Fig 16 of [1] that even manual code starts to fail at 10M EMF elements in a 4GB VM. A simple EMF activity such as EcoreUtil copy may take nearly 10 seconds on 10M elements on an ordinary PC. Poor code may easily go unacceptably quadratic.

In practice there are no doubt many algorithms in Sirius that can be improved, but until then you would be well advised to consider approaches that reduce the number of model elements. Otherwise get a really big fast PC.

Regards

Ed Willink

[1] http://www.eclipse.org/mmt/qvt/docs/ICMT2017/MicromappingMoC.pdf



Hi Ed Willink, Thanks for your reply.
I saw here :http://www.tuicool.com/articles/j2QJRz2.
It Said:
The goal was to get a runtime which react well with 1 million elements. Why one million? Because it is a beautiful number... but also because we noted that the modeling projects are fastly growing to 200K elements and 500K in case of in-depth analysis. Usually when you are over 500K, it means it's time to re-think the process as you will likely need to decouple models with a clear and defined life-cycle. That is why we aim Sirius to work with 1 millions, then it will be really smoothly with the more usual 500K use case.

and it has a table :
Time (sec) Heap (Mb) Time Variation Heap Variation
Open Huge Project 80 276 -31,00% -20,00%
Open Big Class Diagram 11 24 -54,00% +20,00%
Refresh Big Class Diagram 0,731 0 -18,00% 0,00%
Save After Diagram Change 26 0 -23,00% 0,00%
Close Diagram 0,1 -6 0,00% 0,00%
Close Project 55 294 0,00% 0,00%

That means This test was base on 1M elements, and Sirius could open 1M elements.
but when I run this 1M elements, it can't be open, I don't know the reason.
Re: Sirius benchmark [message #1774247 is a reply to message #1774173] Thu, 12 October 2017 08:13 Go to previous messageGo to next message
Cedric Brun is currently offline Cedric BrunFriend
Messages: 431
Registered: July 2009
Senior Member
Hi, thanks for trying to reproduce the benchmark. We are always pushing further from a performance perspective and having a model with 1 Million elements is definitely manageable though require a bit of setup to not go into dead ends, among others: efficient serialization format (not XMI), diagrams which are users constructed and focused on one aspect of the model, small work on the memory layout of the model (see this checklist http://cedric.brun.io/eclipse/ecore-design-checklist-part2/ )

I often see a lot of assumptions which are just completely off the mark regarding performances when we look at the machines we have now and the improvements done in EMF, Sirius and even Java.

I just reproduced the benchmark myself by using a Obeo Designer Community Edition 10.0 which i had on my machine and could reproduce these times. Note that this project is even sub-optimal as having a single binary file holding all the model would probably require less (I would bet on half the time, but that needs to be checked)

My machine is a Lenovo X240 on Linux Opensuse 42.2, Java 1.8.0_73-b02.
I used this archived project: https://bugs.eclipse.org/bugs/attachment.cgi?id=254581

I just uploaded a video showing exactly what I did: https://www.youtube.com/watch?v=qJ4rRpufSVI&feature=youtu.be


http://cedric.brun.io news and articles on eclipse and eclipse modeling.
Re: Sirius benchmark [message #1774255 is a reply to message #1774247] Thu, 12 October 2017 09:16 Go to previous messageGo to next message
Cedric Brun is currently offline Cedric BrunFriend
Messages: 431
Registered: July 2009
Senior Member
> (I would bet on half the time, but that needs to be checked)

I'm lucky I did not bet as it turns out that merging all the model in a single .ecorebin file brings non significant improvements.


http://cedric.brun.io news and articles on eclipse and eclipse modeling.
Re: Sirius benchmark [message #1774267 is a reply to message #1774247] Thu, 12 October 2017 11:45 Go to previous messageGo to next message
Bo Zhang is currently offline Bo ZhangFriend
Messages: 28
Registered: July 2017
Junior Member
Cedric Brun wrote on Thu, 12 October 2017 16:13
Hi, thanks for trying to reproduce the benchmark. We are always pushing further from a performance perspective and having a model with 1 Million elements is definitely manageable though require a bit of setup to not go into dead ends, among others: efficient serialization format (not XMI), diagrams which are users constructed and focused on one aspect of the model, small work on the memory layout of the model (see this checklist http://cedric.brun.io/eclipse/ecore-design-checklist-part2/ )

I often see a lot of assumptions which are just completely off the mark regarding performances when we look at the machines we have now and the improvements done in EMF, Sirius and even Java.

I just reproduced the benchmark myself by using a Obeo Designer Community Edition 10.0 which i had on my machine and could reproduce these times. Note that this project is even sub-optimal as having a single binary file holding all the model would probably require less (I would bet on half the time, but that needs to be checked)

My machine is a Lenovo X240 on Linux Opensuse 42.2, Java 1.8.0_73-b02.
I used this archived project: https://bugs.eclipse.org/bugs/attachment.cgi?id=254581

I just uploaded a video showing exactly what I did: https://www.youtube.com/watch?v=qJ4rRpufSVI&feature=youtu.be


HI Cedric Brun,
Thank you very much, I've seen the video in YouTube.
I know what's the problem when I reproduce the benchmark. I'm try to create a new ServiceInterpreter Diagram in the index package of index.ecore, so I'm faild.

I have some questions about Sirius performance benchmark:
1. I saw there is a little elements shown in the diagram, but my requirement is to show millions of elements in one diagram, is it possible in Sirius? like create a new ServiceInterpreter Diagram in the index package of index.ecore.

2. How many elements could Sirius display in one diagram? Have you test about it?

3. Could Sirius only display part of the huge diagram for improve performance? When user move the window, then compute and display another part?
Re: Sirius benchmark [message #1774270 is a reply to message #1774267] Thu, 12 October 2017 12:11 Go to previous messageGo to next message
Cedric Brun is currently offline Cedric BrunFriend
Messages: 431
Registered: July 2009
Senior Member
Thanks for the details.

1. No it's not possible to show millions of elements in one diagram in Sirius, it has not been designed to support such use cases from a performance and user experience perspective.
2. It might depend on what you consider exactly an "element" in a diagram, but roughly speaking 1000 elements in the diagram is the right order of magnitude.
3. That would require deep changes in Sirius itself, in the underlying technologies and also probably in the way the diagram would be defined. At that point the better way to support such use cases would be to implement a new "Dialect" from scratch (Table, Tree and Diagram supports are dialects) with an editor able to do that.


http://cedric.brun.io news and articles on eclipse and eclipse modeling.
Re: Sirius benchmark [message #1774332 is a reply to message #1774270] Fri, 13 October 2017 03:28 Go to previous messageGo to next message
Bo Zhang is currently offline Bo ZhangFriend
Messages: 28
Registered: July 2017
Junior Member
Cedric Brun wrote on Thu, 12 October 2017 20:11
Thanks for the details.

1. No it's not possible to show millions of elements in one diagram in Sirius, it has not been designed to support such use cases from a performance and user experience perspective.
2. It might depend on what you consider exactly an "element" in a diagram, but roughly speaking 1000 elements in the diagram is the right order of magnitude.
3. That would require deep changes in Sirius itself, in the underlying technologies and also probably in the way the diagram would be defined. At that point the better way to support such use cases would be to implement a new "Dialect" from scratch (Table, Tree and Diagram supports are dialects) with an editor able to do that.


Thank you very much.

For 3, I understand that this would be a quite comlicated task. But I really need this feature. If I want to implement it, how could I define the diagram incrementally when I define the diagram? I mean, is it like I could only load part of the model and display it, when I move to other part of the diagram, the corresponding part of the model would be loaded then? How could I implement this? Could you give some suggestions about where to find such information?

Re: Sirius benchmark [message #1774354 is a reply to message #1774332] Fri, 13 October 2017 08:01 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6989
Registered: July 2009
Senior Member
HI

IIRC Sirius has an ability to populate a diagram from seeds then gradually add neighbours.

You could therefore manually choose distinct nodes in your design that you want to focus on and create a diagram around each of them.

One could imagine a create-multiple-class-diagrams wizard that offered a check list in which the central nodes of each diagram are selected and a neighbourhood distance configured. One could even imagine a heuristic that preselected the central nodes to achieve complete coverage with a near minimal number of central nodes.

Regards

Ed Willink
Re: Sirius benchmark [message #1774366 is a reply to message #1774354] Fri, 13 October 2017 11:16 Go to previous messageGo to next message
Bo Zhang is currently offline Bo ZhangFriend
Messages: 28
Registered: July 2017
Junior Member
Ed Willink wrote on Fri, 13 October 2017 16:01
HI

IIRC Sirius has an ability to populate a diagram from seeds then gradually add neighbours.

You could therefore manually choose distinct nodes in your design that you want to focus on and create a diagram around each of them.

One could imagine a create-multiple-class-diagrams wizard that offered a check list in which the central nodes of each diagram are selected and a neighbourhood distance configured. One could even imagine a heuristic that preselected the central nodes to achieve complete coverage with a near minimal number of central nodes.

Regards

Ed Willink



Is it config the element as not Synchronized?

[Updated on: Fri, 13 October 2017 11:20]

Report message to a moderator

Re: Sirius benchmark [message #1774376 is a reply to message #1774366] Fri, 13 October 2017 12:55 Go to previous message
Cedric Brun is currently offline Cedric BrunFriend
Messages: 431
Registered: July 2009
Senior Member
Yes, you can have a look at EcoreTools which is a "real world" example of such behavior.

http://cedric.brun.io news and articles on eclipse and eclipse modeling.
Previous Topic:Sirius validation problemes
Next Topic:Problem with refresh diagram programmatically
Goto Forum:
  


Current Time: Thu Jan 21 12:37:18 GMT 2021

Powered by FUDForum. Page generated in 0.01742 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top