Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cdt-dev] Help with Indexer Bug

So what is the solution here?

 

Pragma once semantics are only valid within the bounds of the current compilation unit.

 

I can see that when processing a.c, that it is a good assumption that all instances of a include guarded header are going to be the same, but it is not a good assumption that all the headers included in a.c are going to be exactly the same as the headers included in b.c.

 

To me it seems like the index should store version based on the top most level file, so common.h from the a.c tree would be the same, but then when common.h is encountered from the b.c tree, it would be different.

From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Schorn, Markus
Sent: Wednesday, October 02, 2013 9:54 AM
To: CDT General developers list.
Subject: Re: [cdt-dev] Help with Indexer Bug

 

I tried to give an explanation in my previous email:

The attempt to correctly generate multiple versions resulted in way to many header versions, such that the indexing was too slow and the database grew to unreasonable sizes.

Therefore CDT does make the assumption that a header file with pragma once semantics produces the same macro-definitions and declarations whenever it is included within one project.

Markus.

 

From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Joseph Henry
Sent: Wed, 02. 10. 2013 15:40
To: CDT General developers list.
Subject: Re: [cdt-dev] Help with Indexer Bug

 

So here is what I have found:

 

In CPreprocessor.detectIncludeGuard if an include guard is found, then pragmaOnce semantics are turned on, but what is causing this bug, is that trackSignificantMacros is never turned on.

 

Why are significant macros not turned on just because an include guard is found? There are thousands of ways that this could result in the index being wrong.

 

In my opinion, track significant macros should always be turned on.

 

From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Schorn, Markus
Sent: Wednesday, October 02, 2013 2:15 AM
To: CDT General developers list.
Subject: Re: [cdt-dev] Help with Indexer Bug

 

I guess common.h is identified as a file with pragma once semantics, and therefore the index will maintain only a single version of it. To verify this, you can put some useless code before the include-guard. This should cause the indexer to create the two different versions for common.h

 

Why does this work like this?

When implementing the support for multiple versions we ended up with the following situation:

The attempt to correctly generate multiple versions resulted in way to many header versions, such that the indexing was too slow and the database grew to unreasonable sizes.

Therefore CDT does make the assumption that a header file with pragma once semantics produces the same macro-definitions and declarations whenever it is included within one project.

 

So, to overcome the indexing problem you need to find a way that prevents CDT from assigning the pragma once semantics property to this file.

 

Markus.

 

From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Joseph Henry
Sent: Tue, 01. 10. 2013 23:37
To: CDT General developers list.
Cc: CDT General developers list.
Subject: Re: [cdt-dev] Help with Indexer Bug

 

When processing b.c common.h is read from the index, and whatever version is read does not include the definition for SYMBOL.

Sent from my iPhone


On Oct 1, 2013, at 5:14 PM, "Sergey Prigogin" <eclipse.sprigogin@xxxxxxxxx> wrote:

The next step is to verify that two variants of common.h are written to the index. These two variants should be produced while indexing a.c and b.c respectively due to SYMBOL being defined in one case and undefined in the other.

 

-sergey

 

On Tue, Oct 1, 2013 at 1:50 PM, Joseph Henry <Joseph.Henry@xxxxxxx> wrote:

Actually it does seem to be added.

 

In the function CPrepocessor.executeIfdef

 

There is a block of code:

 

macro= fMacroDictionary.get(namechars);

                                                isTaken= (macro == null) == isIfndef;

                                                if (macro == null) {

                                                                macro = new UndefinedMacro(namechars);

                                                                fCurrentContext.significantMacroUndefined(namechars);

                                                } else {

                                                                fCurrentContext.significantMacroDefined(namechars);

                                                }

 

 

It gets added in the else.

From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Sergey Prigogin
Sent: Tuesday, October 01, 2013 4:37 PM


To: CDT General developers list.
Subject: Re: [cdt-dev] Help with Indexer Bug

 

Do you know if SYMBOL is added to significant macros of common.h? If not, what condition prevents it from being added?

 

-sergey

 

On Tue, Oct 1, 2013 at 1:15 PM, Joseph Henry <Joseph.Henry@xxxxxxx> wrote:

Ok,

 

So I have (mostly) solved the problem with the #pragma once. I will be pushing the code change soon, but I have found another indexer issue.

 

Lets say I have file a.h

 

#ifndef A_H_

#define A_H_

 

#define SYMBOL 1

 

#endif

 

And file common.h

 

#ifndef COMMON_H_

#define COMMON_H_

 

#if !defined(SYMBOL)

#define SYMBOL 2

#endif

 

#define COMMON

 

#endif

 

If I have file a.c:

 

#include "a.h"

 

#include "common.h"

 

int* a = SYMBOL;

 

and b.c:

 

#include "common.h"

 

int* b = SYMBOL;

 

b.c will say the SYMBOL is undefined.

 

This does not matter whether or not it is in the form of #ifndef SYMBOL or #if !defined(SYMBOL)

 

 

I will be filling a bug for this shortly, but in the meantime, do you have any ideas what could be causing this?

 

Obviously the indexer thinks that symbol has already been defined, but from b.c, it clearly has not.

From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Sergey Prigogin
Sent: Monday, September 30, 2013 3:25 PM


To: CDT General developers list.
Subject: Re: [cdt-dev] Help with Indexer Bug

 

I agree that presence of any conditions around #pragma once should not prevent detection of #ifdef/#defne/#endif include guard. It's hard for me to give any practical suggesting regarding the code, since I haven't seen it.

 

-sergey

 

On Mon, Sep 30, 2013 at 12:09 PM, Joseph Henry <Joseph.Henry@xxxxxxx> wrote:

Well I did find one case in which this will not work, maybe you can help me with this.

 

So lets say that pragmaInclude.h was changed to this:

 

#if     _MSC_VER > 1000

#pragma once

#endif

 

#ifndef PRAGMAINCLUDE_H_

#define PRAGMAINCLUDE_H_

 

#define SYMBOL 0

 

#endif

 

The code that I wrote will not skip over the #pragma once in this case. The only variables that I have in this function is the Lexer of the code, so I am not really sure how I can ascertain that I need to process this line or not.

 

Any Ideas?

 

From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Sergey Prigogin
Sent: Monday, September 30, 2013 3:01 PM


To: CDT General developers list.
Subject: Re: [cdt-dev] Help with Indexer Bug

 

 

 

On Mon, Sep 30, 2013 at 11:37 AM, Joseph Henry <Joseph.Henry@xxxxxxx> wrote:

Ok,

 

So I have implemented a solution that seems to solve this problem.

 

What I did was in IncludeGuardDetection.findIncludeGuard I added a some logic to skip the first #pragma once (if it exists).

 

This seems to work without messing up anything.

 

What is the next step? How would I go about possibly contributing this fix?

 

This is great that you were able to implement a solution to the problem. Could you please push the patch to Gerrit (http://wiki.eclipse.org/CDT/git#Using_Gerrit_for_CDT) for review. Thanks.

 

Thanks,

Joseph Henry.

-sergey 

 

From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Sergey Prigogin


Sent: Sunday, September 22, 2013 1:42 AM

To: CDT General developers list.
Subject: Re: [cdt-dev] Help with Indexer Bug

 

The attached project is quite more complex than what was discussed in this email thread so far. Is the problem reproducible with the code quoted in this email thread?

 

-sergey

 

On Fri, Sep 20, 2013 at 8:40 PM, Joseph Henry <Joseph.Henry@xxxxxxx> wrote:

It is. I have attached a project that reproduces this case in both bugs named in this thread. 

Sent from my iPhone


On Sep 20, 2013, at 5:02 PM, "Sergey Prigogin" <eclipse.sprigogin@xxxxxxxxx> wrote:

 

 

On Fri, Sep 20, 2013 at 1:47 PM, Joseph Henry <Joseph.Henry@xxxxxxx> wrote:

When CPreprocessor encounters #include "header.h" in b.c it should include content from both, header.h and stddef.h. header.h may be included from the index or as a source.

 

But it does not. When header.h is read from the index, it contains nothing, except what is defined in header.h

 

Lets say that header.h looks like this.

 

#ifndef _HEADER

#define _HEADER

 

#include “stddef.h”

 

#endif

 

And lets say stddef.h looks like this

 

#pragma once

#define NULL 0

 

In the function processInclusionFromIndex in CPrepocessor, the InternalFileContent is read from the index. When it calls getMacroDefinitions(), this returns only 1 definition which is _HEADER.  If the #pragma once is not there this returns 2 definitions which are _HEADER and NULL

 

There seems to be a gap between discussion in this email and the actual reproducible case. Is the problem reproducible with stddef.h qouted in this email?

 

-sergey 

 

From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Sergey Prigogin
Sent: Friday, September 20, 2013 4:07 PM


To: CDT General developers list.
Subject: Re: [cdt-dev] Help with Indexer Bug

 

 

 

On Fri, Sep 20, 2013 at 12:44 PM, Joseph Henry <Joseph.Henry@xxxxxxx> wrote:

Honestly I am not really sure.

 

This is what I saw when stepping though the code in the debugger.

 

File stddef.h has this structure:

 

#pragma once

#ifndef __STD

#define __STD

 

…..

 

#endif

 

I have a File with the following structure:

a.c
     |_ stddef.h
     |_ header.h
         |_ stddef.h  

 

and there is another File with the following structure:



    b.c
     |_ header.h
     |    |_ stddef.h

 

 

Now this is what I know happens:

 

When b.c is parsed, it finds the #include “header.h” and executes that statement.

When it looks for header.h it finds it, and its location is in the index.

Now header.h is included from the index, but it does not contain any instance of stddef.h.

 

A header stored in the index never contains content from another header. Each header is stored separately. When CPreprocessor encounters #include "header.h" in b.c it should include content from both, header.h and stddef.h. header.h may be included from the index or as a source. This depends on significant macros of header.h.

 

-sergey

 

What I don’t understand is why (or where) header.h is stored in the index without any instance of stddef.h. It seems like a #pragma once should be treated just like a #ifndef #define #endif block. So from b.c since this file has not been encountered, it should be included.

 

From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Sergey Prigogin
Sent: Friday, September 20, 2013 3:12 PM


To: CDT General developers list.
Subject: Re: [cdt-dev] Help with Indexer Bug

 

 

 

On Fri, Sep 20, 2013 at 11:22 AM, Joseph Henry <Joseph.Henry@xxxxxxx> wrote:

Ok,

 

After further looking into this, is seems that these are 2 separate issues. If you look at https://bugs.eclipse.org/bugs/show_bug.cgi?id=413768#c2 it shows a use case that supposedly should work, but that is the exact use case that is broken for me (with one minor change). I will explain.

 

Basically I have this situation:

 

    a.c

     |_ stddef.h  (version S1)

     |_ header.h

         |_ stddef.h  (version S2)

         |_ use declaration found in S1

 

And these another source file that contains header.h

 

    b.c

     |_ header.h

     |    |_ stddef.h  (version S1)

     |_ use declaration found in S1

 

 

The problem comes into play where stddef.h has a #pragma once in it at the top of the file like:

 

If stddef.h had #pragma once, there would not be S2, would it?

 

Did you mean something like 

 

a.c
     |_ stddef.h  (version S1)
     |_ header.h
         |_ stddef.h  (version S1)
         |_ use declaration found in S1

    b.c
     |_ header.h
     |    |_ stddef.h  (version S2)
     |_ use declaration found in S2

 

?

 

#pragma once

#ifndef __STDDEF__

#define __STDDEF__

 

…….

#define NULL 0

……

 

#endif

 

The #pragma once is what is causing the problem. Since it is outside of the #ifndef block, it seems like the version of stddef.h that is included in header.h is just simply not put in the index. So from b.c, the definition of NULL cannot be found.

 

Maybe this explains my problem a bit more.

 

Any help would be great.

 

Thanks,

Joseph Henry.

 

-sergey 

 

From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Sergey Prigogin
Sent: Friday, September 20, 2013 1:08 PM
To: CDT General developers list.
Subject: Re: [cdt-dev] Help with Indexer Bug

 

https://bugs.eclipse.org/bugs/show_bug.cgi?id=413768#c11 contains analysis of the problem and suggests a potential solution. I'm not sure whether that solution would not break other use cases, but it's at least something to try. I would start with diagnosing the problem described in bug 417193 similar to how it wasdone for bug 413768.

 

-sergey

 

On Fri, Sep 20, 2013 at 9:50 AM, Joseph Henry <Joseph.Henry@xxxxxxx> wrote:

Hi all,

 

I have filed bug 417193, which seems to be related to bug 413768.

 

I have not gotten much response on either of these bugs and was wondering if someone could point me in the right direction to fix this. I really need this bug fixed so I have been looking into it, but cant seem to make any headway.

 

If anyone could point me in the right direction that would be great.


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev

 


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev

 


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev

 


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev

 

_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev

 


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev

 


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev

 


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev

 


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev

 

_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev


Back to the top