Re: [jdt-dev] Performance degradation switching from 1.8 to 11

[Andrey Loskutov <loskutov@xxxxxx>]

Hi Gunnar,

I would prefer 3), but I would first apply your O(N) -> O(1) patch, because the bigger refactoring could take lot of time / cause lot of side effects.

Please also create a new dedicated bug for moving JDT from old *Tables to standard collections API.

Despite the memory is cheap today, for 3) it would be valuable to know how much it would cost, especially for "fat" workspaces with ~ 100.000+ Java files.

I would not care if the memory use would rise from 2 GB to 2.2 GB (10%), but if that would be 2 -> 3 GB (50%), I would probably vote against it.

We (internally) currently run Eclipse with 12 / 31 / 48 GB heap by default, depending on workstations RAM (32/64/256 GB), so I would sleep better if that still would be enough for the patched JDT :-) 

Kind regards,
Andrey Loskutov

Спасение утопающих - дело рук самих утопающих

https://www.eclipse.org/user/aloskutov


> Gesendet: Dienstag, 29. September 2020 um 20:46 Uhr
> Von: "Gunnar Wagenknecht" <gunnar@xxxxxxxxxxxxxxx>
> An: "Eclipse JDT general developers list." <jdt-dev@xxxxxxxxxxx>
> Betreff: Re: [jdt-dev] Performance degradation switching from 1.8 to 11
>
> I have a general question while I was working on this. 
> 
> I did a couple of changes by replacing the implementation of those classes. I discovered some "convenience" methods which are different that the contacts in standard collections (eg., #put returning the *new* value instead of the existing). I kept those to not cause too much changes for now.
> 
> What is the JDT team's preference here? We have a couple options.
> 
> (1) Hide HashMap inside existing implementations
> 
> Example:
> https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/169915/2/org.eclipse.jdt.core/compiler/org/eclipse/jdt/internal/compiler/util/HashtableOfPackage.java
> 
> - callers need to be changed if they access underlying arrays directly
> - additional indirection is added
> 
> (2) Make existing implementation extend HashMap
> 
> Example:
> https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/170026/1/org.eclipse.jdt.core/compiler/org/eclipse/jdt/internal/compiler/util/HashtableOfType.java
> 
> - callers need to be changed if they access underlying arrays directly
> + class becomes a drop-in replacement for collections with convenient methods
> - callers have more flexibility (eg., stream api) but need to pay attention to details (eg., arrays keys wrapped into key object)
> 
> (3) Remove the existing implementation entirely
> 
> - we eliminate the legacy classes
> - all callers need to be switched to collections api
> - still need to introduce key object for arrays (for the desired hashCode/equals implementation)
> 
> 
> My internal testing confirmed that they improve the situation at scale. Also, it looks like those modifications to not regress JDT performance for the default users. There is a cost of memory, which I haven't measured tbh and don't really think it's an important goal these days (compared to performance).
> 
> 
> However, to be fair, the pure performance boost in my case does not come from those modifications. It's a "simple" loop optimization from O(N IPackageFragmentRoot.total.on.classpath) to O(1 map-lookup + result.size).
> https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/170035/1
> 
> 
> Should I continue with those efforts of changing the data structures or is this not something the JDT team is looking at adopting?
> 
> 
> Thoughts?
> 
> -Gunnar