[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
RE: [cdt-dev] Performance issues with indexer/refresh on large scale project
|
Thanks for these remarks. So, at the moment, I'm lost with
this :-( As a last resort, I can hide folders on the ClearCase level, but that's
not exactly what I want. (It#s nice to have the test folder hidden in the DIE,
but it's not so nice if is no longer available on the command line
either...)
I had a look at the bug report Tim mentioned, https://bugs.eclipse.org/bugs/show_bug.cgi?id=181020. I
guess even if that was solved, it wouldn't help us too much, as we have much
more folders to include than to exclude. It would become quite a hassle to setup
the project to include only the needed folders (~200) instead of excluding the 5
unwanted ones.
The flexible resource feature will allow to exclude
specific parts of the source tree, Doug?
BTW: any pointer to my other two questions? Thanks
again!
Achim
>>Just to
clarify, the flexible resource feature that was done for e4 is being contributed
to Eclipse 3.6
Oh, thanks for clarifying : )
On Tue, Jan 26, 2010 at 11:18 AM,
<Tim.Kelly@xxxxxxxxx>
wrote:
Some
things we found when doing performance analysis/benchmarking
in a similar scenario (the general case for
Nokia developers):
1)
Import time is linear WRT # of source files under the root.
After
about 15K source files under the root it took a couple minutes to import a
project.
At about
25K source files is took almost 5 minutes. We decided import was the time
you click Finish on a project creation wizard until you can open a source file from
the Project Explorer.
We
added a timer to
our project creation action posted the project creation time to
the error log the same way the Indexer does (i.e. Indexed '<project>'
....) so we could monitor problematic projects.
The
majority of time importing is spent querying the file system for file
attribs & directory sublists (refreshing is actually quite fast though
to the user it appears to be a a lingering job)
2)
Indexing time is related to # of file under the source roots. We
introduced the concept of nested source roots but I see the target milestone
was set to 7.0.
However,
for you case of 40K source files, I think you will need e4 (as Doug
already mentioned). I found any project with that many source
files under the root makes the IDE about
unusable.
Just to clarify, the flexible resource feature that was done for e4
is being contributed to Eclipse 3.6. So you don't need e4, but you will need
the next release of the Platform.
On Tue, Jan 26, 2010 at 10:07 AM, Bursian, Achim
<achim.bursian@xxxxxxxxxxx> wrote:
Hi all,
I'm running into serious performance
problems with a ~40k files C++ project (standard make project on
Solaris/Sparc with ClearCase as SCC). Importing and indexing the project
takes hours, which is at least partly caused by a lot of unnecessary
files being opened.
Three questions:
* How can I programmatically disable
the automatic indexing for a period of time, and reenable it
later (I found that it's faster to do a refresh without the indexer
running, and doing a full indexer run afterwards)? I'd try to do that in
a startup plugin, which would disable indexing, import the project,
refresh everything, then run the indexer. Maybe that could speed up
things...
* I would like to exclude a complete
part of the project hierarchy from being seen by Eclipse and CDT. I
thought the "filter" setting ("Source folder exclusion pattern") on the
CDT source folders property page would do just that, but no. I
excluded the "test/" folder and some others with the filter, now it
is no longer shown in the project explorer. But monitoring the system
with Solaris dtrace shows that the whole <projectroot>/test
folder hierarchy is still scanned (stat system call) during a refresh,
and a lot of files there even get opened (Imakefiles, *.txt, *.xml).
This is quite bad, as it contains 20k files that are only testcases and
data stuff. Anything I can do?
This is actually the Eclipse Platform refreshing your project,
not the CDT. This is one of the main drivers for the Flexible Resources
feature coming in this year's train release, to cut down on what Eclipse
sees in a project directory tree.
* Is there any general guideline for
setting up such a large scale project with CDT,
anywhere?
Thanks!
Achim
_______________________________________________
cdt-dev
mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev
mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
