[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
RE: [cdt-dev] Major PDOM changes in HEAD
|
Hmm, now that I've met my performance targets using caching, maybe I'll put
it back to PDOM per project. The other problem I ran into was the amount of
of memory mapped chunks allowed under Windows which I addressed by making
the chunks bigger.
I have to agree that this would make the reuse feature easier. I'll sleep on
it but if you have a strong opinion let me know. The good news is that it
should be easy to change back (and is a good test of my architecture :).
Doug Schaefer, QNX Software Systems
Eclipse CDT Project Lead, Tools PMC member
http://cdtdoug.blogspot.com
-----Original Message-----
From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On
Behalf Of Doug Schaefer
Sent: Friday, May 19, 2006 8:39 AM
To: CDT General developers list.
Subject: RE: [cdt-dev] Major PDOM changes in HEAD
I don't think I'm doing anything to preclude these requirements. The PDOM is
flexible and theoretically you can run the indexer and create a separate
database file.
The issue with multiple databases will be how to link all the data
structures together. The PDOM fills in as an underlying structure during
fast DOM parses and access to this data needs to be fast. I imagine there
are ways to do it, but I think you'll always need one root file per
workspace as the starting point.
Doug Schaefer, QNX Software Systems
Eclipse CDT Project Lead, Tools PMC member
http://cdtdoug.blogspot.com
-----Original Message-----
From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On
Behalf Of Ed Warnicke
Sent: Friday, May 19, 2006 4:59 AM
To: CDT General developers list.
Subject: RE: [cdt-dev] Major PDOM changes in HEAD
We are speaking about something quite similar. My thought was
to distribute the offline-index in the project with the source code from
the SCM as you suggest. You are correct that this basically entails
either two dbs (the local changes + the version that comes
from the SCM) or db merging (which just *sounds* ugly and painful) :)
Ed
On Fri, 2006-05-19 at 10:48 +0200, Schorn, Markus wrote:
> Let's try to figure out whether we are talking about the same thing.
> What I
> like to have is:
> (1) create an index for several projects on some machine.
> (2) use the offline-index(es) in multiple workspaces.
> (3) once you have the offline-index you still want to locally index
> those
> files that have been changed locally in the workspace.
>
> To make (1) and (2) happen it is helpful to work on a per-project basis,
> as
> every workspace can contain a different set of projects. It's hard to
> make assumptions on what projects are finally combined in an arbitrary
> workspace,
> I'd like to avoid the assumption that the offline-indexes for all
> projects have to come from the same machine/location.
>
> In our commercial product we store the offline-indices in the project
> with the
> source-code and share them via the same mechanisms as the source-code
> (i.e. an
> arbitrary SCM-Tool). This could be an approach that we can follow in
> CDT. Can you share your idea of pushing the offline-index to other
> workspaces?
>
> Solving (3) is a little bit harder and I defer discussing this until I
> have
> understood your ideas on (1) and (2).
>
> Markus.
>
>
>
>
> > -----Original Message-----
> > From: cdt-dev-bounces@xxxxxxxxxxx
> > [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Ed Warnicke
> > Sent: Freitag, 19. Mai 2006 08:31
> > To: CDT General developers list.
> > Subject: RE: [cdt-dev] Major PDOM changes in HEAD
> >
> > I'm not sure I follow you. The model I was considering was
> > having a central service that does builds the database, with
> > the result being pushed out to the team.
> >
> > Ed
> > On Fri, 2006-05-19 at 08:22 +0200, Schorn, Markus wrote:
> > > Hmm, if you do offline indexing you really also want to be
> > able to share
> > > the results within a team. For that you have to be able to combine
> > > multiple
> > > databases in your queries.
> > > Markus.
> > >
> > > > -----Original Message-----
> > > > From: cdt-dev-bounces@xxxxxxxxxxx
> > > > [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Doug Schaefer
> > > > Sent: Freitag, 19. Mai 2006 06:30
> > > > To: CDT General developers list.
> > > > Subject: RE: [cdt-dev] Major PDOM changes in HEAD
> > > >
> > > > Hi Ed,
> > > >
> > > > This shouldn't prohibit off-line indexing. The database
> > > > doesn't really know
> > > > about projects or workspaces. It only knows about files and
> > > > the symbols,
> > > > macros, inclusions, amongst other things that are contained
> > > > in those files.
> > > > This should make it easy to copy information from one
> > > > database to another.
> > > >
> > > > Now, of course the off-line indexing will need to be done
> > > > with a headless
> > > > Eclipse instance with the appropriate Application extension
> > > > to run it. But
> > > > more thought and work needs to be put in it to figure out the
> > > > right way to
> > > > do this.
> > > >
> > > > Doug Schaefer, QNX Software Systems
> > > > Eclipse CDT Project Lead, Tools PMC member
> > > > http://cdtdoug.blogspot.com
> > > >
> > > > -----Original Message-----
> > > > From: cdt-dev-bounces@xxxxxxxxxxx
> > > > [mailto:cdt-dev-bounces@xxxxxxxxxxx] On
> > > > Behalf Of Ed Warnicke
> > > > Sent: Thursday, May 18, 2006 9:32 PM
> > > > To: CDT General developers list.
> > > > Subject: RE: [cdt-dev] Major PDOM changes in HEAD
> > > >
> > > > Doug,
> > > > Doesn't a DB per workspace sort of kill any future hopes
> > > > of pre generating the DB offline? I'm not talking about
> > > > remoting here,
> > > > just plain old fashion prepopulating...
> > > >
> > > > Ed
> > > > On Thu, 2006-05-18 at 21:26 -0400, Doug Schaefer wrote:
> > > > > Actually, the PDOM was always in the .metadata directory.
> > > > Now, instead of
> > > > > having one database per project, there is only one database
> > > > per workspace.
> > > > >
> > > > > And because the PDOM is memory mapped into the Eclipse
> > > > process, it has to
> > > > be
> > > > > local. Right now I'm having enough trouble squeezing reasonable
> > > > performance
> > > > > out of a strictly local architecture. I have no idea how to
> > > > properly do
> > > > > remote projects. For that, I'll need some help :).
> > > > >
> > > > > Doug Schaefer, QNX Software Systems
> > > > > Eclipse CDT Project Lead, Tools PMC member
> > > > > http://cdtdoug.blogspot.com
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: cdt-dev-bounces@xxxxxxxxxxx
> > > > [mailto:cdt-dev-bounces@xxxxxxxxxxx] On
> > > > > Behalf Of Greg Watson
> > > > > Sent: Thursday, May 18, 2006 9:07 PM
> > > > > To: CDT General developers list.
> > > > > Subject: Re: [cdt-dev] Major PDOM changes in HEAD
> > > > >
> > > > > Doug,
> > > > >
> > > > > I know this is a bit left of base, bit I wonder if this
> > change is
> > > > > going to make support for remote projects more
> > difficult? I had
> > > > > imagined that if the PDOM database was stored in the project
> > > > > directory then it should be able to be generated on the remote
> > > > > machine, but still be visible to Eclipse running on the local
> > > > > machine. If the database is stored at the workspace
> > level then I
> > > > > don't think this approach will work any more (unless the whole
> > > > > workspace is remote.)
> > > > >
> > > > > Just a thought.
> > > > >
> > > > > Greg
> > > > >
> > > > >
> > > > > On May 18, 2006, at 5:47 PM, Doug Schaefer wrote:
> > > > >
> > > > > > Hey gang,
> > > > > >
> > > > > >
> > > > > >
> > > > > > I have made some major last minute changes to improve PDOM
> > > > > > performance. First, the PDOM database is now per
> > > > workspace instead
> > > > > > of per project. This will allow projects to share
> > > > indexing results
> > > > > > with each other. It may also lead to weird results
> > but, for the
> > > > > > moment, it is worth it.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Also, I have done some caching so that the indexer
> > accesses the
> > > > > > PDOM less often during a parse. This has given me a
> > > > Firefox index
> > > > > > time of 13 minutes compared to 53 minutes in CDT
> > 3.0.2. This may
> > > > > > fluctuate as I spend the rest of this release and 3.1.1
> > > > making it
> > > > > > more accurate, but it is a huge step forward.
> > > > > >
> > > > > >
> > > > > >
> > > > > > At any rate, I apologize for the late churn and
> > promise to make
> > > > > > sure indexing and the features that use the index remain
> > > > robust for
> > > > > > our GA release.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > Doug Schaefer, QNX Software Systems
> > > > > >
> > > > > > Eclipse CDT Project Lead, Tools PMC member
> > > > > >
> > > > > > http://cdtdoug.blogspot.com
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > _______________________________________________
> > > > > > cdt-dev mailing list
> > > > > > cdt-dev@xxxxxxxxxxx
> > > > > > https://dev.eclipse.org/mailman/listinfo/cdt-dev
> > > > >
> > > > > _______________________________________________
> > > > > cdt-dev mailing list
> > > > > cdt-dev@xxxxxxxxxxx
> > > > > https://dev.eclipse.org/mailman/listinfo/cdt-dev
> > > > > _______________________________________________
> > > > > cdt-dev mailing list
> > > > > cdt-dev@xxxxxxxxxxx
> > > > > https://dev.eclipse.org/mailman/listinfo/cdt-dev
> > > > _______________________________________________
> > > > cdt-dev mailing list
> > > > cdt-dev@xxxxxxxxxxx
> > > > https://dev.eclipse.org/mailman/listinfo/cdt-dev
> > > > _______________________________________________
> > > > cdt-dev mailing list
> > > > cdt-dev@xxxxxxxxxxx
> > > > https://dev.eclipse.org/mailman/listinfo/cdt-dev
> > > >
> > > _______________________________________________
> > > cdt-dev mailing list
> > > cdt-dev@xxxxxxxxxxx
> > > https://dev.eclipse.org/mailman/listinfo/cdt-dev
> > _______________________________________________
> > cdt-dev mailing list
> > cdt-dev@xxxxxxxxxxx
> > https://dev.eclipse.org/mailman/listinfo/cdt-dev
> >
> _______________________________________________
> cdt-dev mailing list
> cdt-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
