 Expensive @NoDuplicates [message #1096385] |
Wed, 28 August 2013 10:13  |
 Hazem ELRAFFIEE
Messages: 61
Registered: September 2012 |
Member |
|
|
Hello All
I've been using @NoDuplicates a lot in my Sapphire XML Editor, for any field that I find applicable. After while I found out that using my Editor to open some large files, consumes lot of expensive time.
For a 0.5 MB file, it costed me around 7 minutes just to launch!
When I commented all occurrences of @NoDuplicates from my code, I was surprised that the 7 minutes were reduced to only 23 seconds.
Then I decided to implement a Validation Service that simply loops over the list, and see if This item is repeated or not, and returns an Error if found twice.
In order to make it more generic, I created an element called "INamedModelElement" that extends "IModelElement", and contains 1 property called "Name".
Then:
public final class NoDuplicatesValidationService extends ValidationService
{
public Status validate()
{
final INamedModelElement item = context( INamedModelElement.class );
final ModelElementList<INamedModelElement> items = item.nearest( ModelElementList.class );
final IRoot doc = item.nearest( IRoot.class );
if( doc != null )
{
String myName = item.getName().getText();
boolean foundOnce = false;
if(myName != null){
for(INamedModelElement named_item: items){
String hisName = named_item.getName().getText();
if(stringsEqual(myName,hisName)){
if(foundOnce){
return Status.createErrorStatus("No Duplicates Allowed!");
}else{
foundOnce = true;
}
}
}
}
}
return Status.createOkStatus();
}
private static boolean stringsEqual(String myName, String hisName) {
if(hisName == null){
return false;
}
return myName.equalsIgnoreCase(hisName);
}
}
As you can see, for my solution to work, I have to make the element interface extends "INamedModelElement" instead of simply "IModelElement". I also HAVE TO add the "@Service(impl = NoDuplicatesValidationService.class)" at the "Name" property specifically.
The reason I did all this and not simply keep using "@NoDuplicates", is that when I did so, the 7 minutes were reduced to 26 seconds .. only 3 seconds more than the one with no checking at all, and more than 6 minutes less than "@NoDuplicates".
I'm using Sapphire 0.5, and tried migration to Sapphire 0.6 but that didn't help.
After all I have 2 questions:
- Why does @NoDuplicates take so much time for large files, while it should in fact only do a For Loop?
- For my solution, what if I need to replace @NoDuplicate for a field that is not named as "Name"? Is there any other solution other than creating Another Validation Service?
Thanks!
|
|
|
|
|
|
|
| Re: Expensive @NoDuplicates [message #1102639 is a reply to message #1099190] |
Fri, 06 September 2013 00:26  |
Konstantin Komissarchik
Messages: 1077
Registered: July 2009 |
Senior Member |
|
|
You should plan on moving up to 0.7 development builds. I've been doing further performance and scalability work on this scenario in 0.7 that cannot happen in 0.6.x due to needed API changes. I will push my changes shortly, but the results are very encouraging. I implemented a list index that eliminates the quadratic performance degradation of @NoDuplicates with the number of entries.
Scalability...
In 0.6.4, the tests pass with 100 list entries, but fail with 1000 (stack overflow).
In 0.7 (before index), the tests pass with 1000 list entries, but take too long with 10,000 (minutes).
In 0.7 (after index), the tests pass with 100,000 list entries and will likely pass with higher counts if given enough memory.
Performance...
In 0.7 (before index), the test case with 1000 entries runs in approximately 1.35 seconds, while the test case with 10,000 entries takes many minutes.
In 0.7 (after index), the test case with 1000 entries runs in approximately 0.24 seconds, 10,000 entries in 2.2 seconds, 100,000 entries in 17.2 seconds. The progression is linear.
|
|
|
|
|
|
|
|
|
|
|
|
Powered by
FUDForum. Page generated in 0.04231 seconds
] 
Search
Help
Register
Login
Home
Thank you