Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cdt-dev] Help with Indexer Bug

On Wed, Oct 2, 2013 at 9:46 AM, Joseph Henry <Joseph.Henry@xxxxxxx> wrote:
Adding a comment that Brandon Philips sent to me directly:

Could CDT store in the database the preprocessor symbols actually referenced by each header it parses?

This is exactly what significant macros are.
 Then CDT can check if those specific values have changed to decide if re-parsing is necessary.

Reparsing does happen currently but for performance reasons is limited to to files without include guards or #pragma once.  

 Maybe in the realm of "not a simple change" though. :)

(Sorry to jump in randomly.)


-----Original Message-----
From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Andrew Eidsness
Sent: Wednesday, October 02, 2013 12:34 PM
To: cdt-dev@xxxxxxxxxxx
Subject: Re: [cdt-dev] Help with Indexer Bug

On 13-10-02 12:01 PM, Joseph Henry wrote:
> What was the fairly larger project that you tested?
> I ran a test of my own. The first one is I added a ctx.trackSignificantMacros(); statement in the detectIncludeGuard function in CPreprocessor even if an include guard was found. This fixes all of my issues. This is the output of the log:
> !MESSAGE Indexed 'Index1' (10 sources, 242 headers) in 3.45 sec:
> 31,198 declarations; 65,703 references; 0 unresolved inclusions; 0
> syntax errors; 0 unresolved names (0.00%)
> Now this is how the output without my change:
> !MESSAGE Indexed 'Index1' (10 sources, 221 headers) in 3.18 sec:
> 26,224 declarations; 56,348 references; 0 unresolved inclusions; 101
> syntax errors; 323 unresolved names (0.39%)
> I do not see a significant performance increase, yet I do see that my project resolves perfectly.

The sample projects are just from internal development.  I'm not at all familiar with the structure of the projects, so I can't make any guesses at why one was so much worse.  However, it also happened to be the largest of my three samples.

I don't have the full results from the test anymore, but here is the summary of the indexer (ran just now) from the project that was 7x slower:

C/C++ Indexer: Project 'seven_times_slower' (504 sources, 5844 headers)
    Options: indexer='PDOMFastIndexer', parseAllFiles=true, unusedHeaders=useCPP, skipReferences=false, skipImplicitReferences=false, skipTypeReferences=false, skipMacroReferences=false.
    Database: 119873536 bytes
    Timings: 1342211 total, 319891 parser, 9345 resolution, 51686 index update.
    Errors: 0 internal, 0 include, 0 scanner, 0 syntax errors.
    Names: 399371 declarations, 1735121 references, 350(0.02%) unresolved.
    Cache[64MB]: 4872643790 hits, 7259(0.00%) misses.
Indexer: completed PDOMRebuildTask[1342246ms]

I wouldn't call this project large, but it was bigger the other toy projects that I was using for testing.

Also, I would caution against the change you described.  I was not able to make that variation work without introducing new problems in other projects.

Finally, a technique that I found useful was to put print statements in the constructors for PDOMFile, PDOMMacro, and PDOMInclude.  I used the record id as a unique identifier.  In PDOMInclude I printed both the containing and the target files.  Printing the lists of significant and internallyModified macros generated ALOT of output, so I used filtering and the focused on only one or two at a time.

While looking for these results I found notes on a related problem that I had meant to raise as a bug.  I've done that now (418533).  From what I remember, the description of that bug was one of the things that wound up introducing new bugs when I made the change you described.

cdt-dev mailing list

cdt-dev mailing list

Back to the top