Achim,
That could be the case, I don’t know for sure. I recommend that you disable the preference and see how much it speeds up the indexer, this would either falsify
or verify my conjecture.
Markus.
From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of Achim Bursian
Sent: Montag, 30. Mai 2011 12:13
To: CDT General developers list.
Subject: Re: [cdt-dev] Indexer statistics explained
Importance: Low
Markus,
thanks for your help.
'Skip implicit references' would disable indexing of our operator-> implementation as well, I guess? So it's not really practical, as we make heavy use of smart pointers (body/handle pattern).
Do I understand you correctly, the time used to resolve user defined operators is not included in the parser, resolution and index updates times shown by the indexer?
-Achim
Schorn, Markus wrote on 2011-05-30 11:59:
Hi Achim,
There are a bunch of questions, I respond to the overall performance of the indexer in your project.
My guess is, that in your specific project the resolution of user-defined operators takes quite some time. Your tracing macro adds 12 of them each time you
use it. I had created a few bugs on the performance issue and also worked on improvements, nevertheless it remains to be a rather expensive computation.
To speed up the indexer you can try to disable indexing of user-defined operators (Indexer option ‘Skip implicit references’). It may also help to update CDT
to version 7.0. There are no further performance improvements for resolving user-defined operators after that.
Markus.
Hello,
currently I'm in the process of digging into the indexing process for our huge legacy project. I enabled the indexer statistics, but I'm stuck with some figures:
C/C++ Indexer: Project 'IDXTST' (2154 sources, 2888 headers)
Options: indexer='PDOMFastIndexer', parseAllFiles=false, unusedHeaders=skip, skipReferences=false, skipImplicitReferences=false, skipTypeReferences=false, skipMacroReferences=false.
Database: 108953600 bytes
Timings: 1127585 total, 314138 parser, 58757 resolution, 244067 index update.
Errors: 0 internal, 21 include, 204 scanner, 137 syntax errors.
Names: 492543 declarations, 2420523 references, 24306(0.83%) unresolved.
Cache[88mb]: 264220629 hits, 948(0.00%) misses.
As you see, it takes 1127 seconds to rebuild the index, almost 20 minutes. If I add up the values for parser, resolution and index update, I get only 617 seconds. What happens in the remaining 511 seconds?
Well, now it gets really weird: Indexing the same project, but changing one preprocessor macro which enables some internal tracing code, and the result is as follows:
C/C++ Indexer: Project 'IDXTST' (2153 sources, 2891 headers)
Options: indexer='PDOMFastIndexer', parseAllFiles=false, unusedHeaders=skip, skipReferences=false, skipImplicitReferences=false, skipTypeReferences=false, skipMacroReferences=false.
Database: 122183680 bytes
Timings: 8242961 total, 460663 parser, 70149 resolution, 308092 index update.
Errors: 0 internal, 21 include, 204 scanner, 140 syntax errors.
Names: 498832 declarations, 2870298 references, 25360(0.75%) unresolved.
Cache[88mb]: 652516848 hits, 2077(0.00%) misses.
The total time explodes, it is 8243 seconds now, 2 hours and 17 minutes. All the other numbers increase in a reasonable amount, but total time is way off what I would expect. Here is a table that shows it clearly:
NOTRACES with Traces Factor
sources 2154 2153 1,000
headers 2888 2891 1,001
DB bytes 108953600 122183680 1,121
Timing
total (seconds) 1128 8243 7,310
parser 314 461 1,466
resolution 59 70 1,194
index upd. 244 308 1,262
parser+res+idxupd 617 839 1,360
Diff to total 511 7404
45% 90%
Names:
declarations 492543 498832 1,013
references 2420523 2870298 1,186
unresolved 24306 25360 1,043
In this case, 90% of the total time are not accounted for in the parser, resolution and index updates figures. Why does it increase so dramatically?
Our tracing macro is defined as follows:
#ifdef NOTRACES
#define COCOUT(X,Y,Z)
#else
#define COCOUT(_INDEX,_MASK,_PARS) \
((void)((COTraceFlags[_INDEX]&_MASK) && \
(cout << coutPrep()) _PARS ))
#endif
It is used 34,000 times in the code. Additionally, there are 3,800 places with conditional code inside #ifndef NOTRACES, which effectively is formatting stuff for the trace facility, like this:
#ifndef NOTRACES
COCOUT(TraceDpa,DPA_AC_CONNECTION,
<< "sdDptVarResCallbackFct: "
<< " PDU. len = " << pduLen
<< " Data: " << hex);
for (int i = 0; i < pduLen; i++)
COCOUT(TraceDpa,DPA_AC_CONNECTION, << " 0x" << (int)s7PduP[i]);
COCOUT(TraceDpa,DPA_AC_CONNECTION, << dec << endl);
#endif // !NOTRACES
Any insight into these figures (Markus maybe??).
For reference: We are running Eclipse 3.5.1 with CDT 6.0.1 (yes, there is a very conservative deployment policy in our department).
Thanks, Achim
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev