Expensive @NoDuplicates [message #1096385] |
Wed, 28 August 2013 10:13  |
Eclipse User |
|
|
|
Hello All
I've been using @NoDuplicates a lot in my Sapphire XML Editor, for any field that I find applicable. After while I found out that using my Editor to open some large files, consumes lot of expensive time.
For a 0.5 MB file, it costed me around 7 minutes just to launch!
When I commented all occurrences of @NoDuplicates from my code, I was surprised that the 7 minutes were reduced to only 23 seconds.
Then I decided to implement a Validation Service that simply loops over the list, and see if This item is repeated or not, and returns an Error if found twice.
In order to make it more generic, I created an element called "INamedModelElement" that extends "IModelElement", and contains 1 property called "Name".
Then:
public final class NoDuplicatesValidationService extends ValidationService
{
public Status validate()
{
final INamedModelElement item = context( INamedModelElement.class );
final ModelElementList<INamedModelElement> items = item.nearest( ModelElementList.class );
final IRoot doc = item.nearest( IRoot.class );
if( doc != null )
{
String myName = item.getName().getText();
boolean foundOnce = false;
if(myName != null){
for(INamedModelElement named_item: items){
String hisName = named_item.getName().getText();
if(stringsEqual(myName,hisName)){
if(foundOnce){
return Status.createErrorStatus("No Duplicates Allowed!");
}else{
foundOnce = true;
}
}
}
}
}
return Status.createOkStatus();
}
private static boolean stringsEqual(String myName, String hisName) {
if(hisName == null){
return false;
}
return myName.equalsIgnoreCase(hisName);
}
}
As you can see, for my solution to work, I have to make the element interface extends "INamedModelElement" instead of simply "IModelElement". I also HAVE TO add the "@Service(impl = NoDuplicatesValidationService.class)" at the "Name" property specifically.
The reason I did all this and not simply keep using "@NoDuplicates", is that when I did so, the 7 minutes were reduced to 26 seconds .. only 3 seconds more than the one with no checking at all, and more than 6 minutes less than "@NoDuplicates".
I'm using Sapphire 0.5, and tried migration to Sapphire 0.6 but that didn't help.
After all I have 2 questions:
- Why does @NoDuplicates take so much time for large files, while it should in fact only do a For Loop?
- For my solution, what if I need to replace @NoDuplicate for a field that is not named as "Name"? Is there any other solution other than creating Another Validation Service?
Thanks!
|
|
|
Re: Expensive @NoDuplicates [message #1096826 is a reply to message #1096385] |
Wed, 28 August 2013 23:06   |
Eclipse User |
|
|
|
There are two parts to @NoDuplicates implementation: (a) validation logic and (b) trigger for when to re-validate. Your implementation is just (a). The performance issue you are seeing is likely caused by unnecessary re-validation. Each property should not be validated more than once on launch.
Could you open a bug and I will investigate further?
|
|
|
|
|
Re: Expensive @NoDuplicates [message #1099190 is a reply to message #1098144] |
Sun, 01 September 2013 09:08   |
Eclipse User |
|
|
|
I can see that my implementation doesn't always re-validate when it needs to re-validate!
I'll try 0.6.4 as fast as I can, may be this week. That will save me a lot!
For the items count I used, it was of different categories, all using @NoDuplicates:
Type1: 301 items
Type2: 186 items
Type3: 29 items
Type4: 408 items
|
|
|
Re: Expensive @NoDuplicates [message #1102639 is a reply to message #1099190] |
Fri, 06 September 2013 00:26   |
Eclipse User |
|
|
|
You should plan on moving up to 0.7 development builds. I've been doing further performance and scalability work on this scenario in 0.7 that cannot happen in 0.6.x due to needed API changes. I will push my changes shortly, but the results are very encouraging. I implemented a list index that eliminates the quadratic performance degradation of @NoDuplicates with the number of entries.
Scalability...
In 0.6.4, the tests pass with 100 list entries, but fail with 1000 (stack overflow).
In 0.7 (before index), the tests pass with 1000 list entries, but take too long with 10,000 (minutes).
In 0.7 (after index), the tests pass with 100,000 list entries and will likely pass with higher counts if given enough memory.
Performance...
In 0.7 (before index), the test case with 1000 entries runs in approximately 1.35 seconds, while the test case with 10,000 entries takes many minutes.
In 0.7 (after index), the test case with 1000 entries runs in approximately 0.24 seconds, 10,000 entries in 2.2 seconds, 100,000 entries in 17.2 seconds. The progression is linear.
|
|
|
|
|
Re: Expensive @NoDuplicates [message #1107440 is a reply to message #1106172] |
Thu, 12 September 2013 11:59   |
Eclipse User |
|
|
|
35 seconds .. I successfully migrated into Sapphire 0.7 latest build @ 10/09/2013, and replaced all occurrences of my NoDuplicatesValidationService with @NoDuplicates .. and I reached 35 seconds for the same file.
That's great for me, what do you think?
|
|
|
|
Re: Expensive @NoDuplicates [message #1109936 is a reply to message #1107642] |
Mon, 16 September 2013 06:22   |
Eclipse User |
|
|
|
Hey Konstantin,
We've been following the 0.7.x builds and when we use @NoDuplicates with a property that in the UI is using CheckBoxListPropertyEditorRenderer$Factory as factory hint, we have been getting the following error. Any ideas?
ERROR : Name property is already disposed.
java.lang.IllegalStateException: Name property is already disposed.
at org.eclipse.sapphire.Property.assertNotDisposed(Property.java:761)
at org.eclipse.sapphire.Property.init(Property.java:83)
at org.eclipse.sapphire.Value.text(Value.java:244)
at org.eclipse.sapphire.Value.text(Value.java:239)
at org.eclipse.sapphire.services.UniqueValueValidationService.isUniqueValue(UniqueValueValidationService.java:91)
at org.eclipse.sapphire.services.UniqueValueValidationService.compute(UniqueValueValidationService.java:77)
at org.eclipse.sapphire.services.UniqueValueValidationService.compute(UniqueValueValidationService.java:1)
at org.eclipse.sapphire.services.DataService.refresh(DataService.java:56)
at org.eclipse.sapphire.services.UniqueValueValidationService$1.handleTypedEvent(UniqueValueValidationService.java:62)
at org.eclipse.sapphire.services.UniqueValueValidationService$1.handleTypedEvent(UniqueValueValidationService.java:1)
at org.eclipse.sapphire.FilteredListener.handle(FilteredListener.java:35)
at org.eclipse.sapphire.ListenerContext$BroadcastJob.run(ListenerContext.java:182)
at org.eclipse.sapphire.ListenerContext.broadcast(ListenerContext.java:120)
at org.eclipse.sapphire.ListenerContext.broadcast(ListenerContext.java:127)
at org.eclipse.sapphire.Property.broadcast(Property.java:709)
at org.eclipse.sapphire.Value.refreshContent(Value.java:145)
at org.eclipse.sapphire.Value.refresh(Value.java:67)
at org.eclipse.sapphire.Value.write(Value.java:414)
at org.eclipse.sapphire.ui.renderers.swt.CheckBoxListPropertyEditorRenderer$Entry.flip(CheckBoxListPropertyEditorRenderer.java:709)
at org.eclipse.sapphire.ui.renderers.swt.CheckBoxListPropertyEditorRenderer$9.checkStateChanged(CheckBoxListPropertyEditorRenderer.java:407)
at org.eclipse.jface.viewers.CheckboxTableViewer$1.run(CheckboxTableViewer.java:212)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
|
|
|
Re: Expensive @NoDuplicates [message #1110129 is a reply to message #1109936] |
Mon, 16 September 2013 12:12   |
Eclipse User |
|
|
|
I have a weird issue as well!!
Inside Eclipse, I had the 35 seconds I just mentioned. But when I exported my editor outside eclipse, it took a very long time to open the file, and still not opening!!
I'm still not able to get an error message, or to know the reason. Once I do, I'll keep you posted.
|
|
|
Re: Expensive @NoDuplicates [message #1110257 is a reply to message #1110129] |
Mon, 16 September 2013 16:13   |
Eclipse User |
|
|
|
Quote:Inside Eclipse, I had the 35 seconds I just mentioned. But when I exported my editor outside eclipse, it took a very long time to open the file, and still not opening!!
That can happen if your JVM is starved for memory. All the time is spent trying to excessively scrub garbage on every memory allocation to prevent an OOM. Check your xmx settings in eclipse.ini file.
|
|
|
Re: Expensive @NoDuplicates [message #1110264 is a reply to message #1110257] |
Mon, 16 September 2013 16:22   |
Eclipse User |
|
|
|
Quote:We've been following the 0.7.x builds and when we use @NoDuplicates with a property that in the UI is using CheckBoxListPropertyEditorRenderer$Factory as factory hint, we have been getting the following error. Any ideas?
There is a similar setup in sapphie-extension.xml editor (service contexts in service definition) and I am not able to reproduce this issue. Judging by the stack trace, you are a bit out of date, so see if newest build will resolve this issue. If not, we will need to dig into what makes your scenario different.
|
|
|
Re: Expensive @NoDuplicates [message #1110661 is a reply to message #1110257] |
Tue, 17 September 2013 06:43  |
Eclipse User |
|
|
|
Konstantin Komissarchik wrote on Mon, 16 September 2013 12:13
That can happen if your JVM is starved for memory. All the time is spent trying to excessively scrub garbage on every memory allocation to prevent an OOM. Check your xmx settings in eclipse.ini file.
Exactly Thank you
|
|
|
Powered by
FUDForum. Page generated in 0.04812 seconds