[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
RE: [cdt-dev] Build Again
|
This is how the projects I work with use
Eclipse/CDT. I had to write a custom Toolchain so that I could hijack the
"scannerInfoCollector" and read the -I/-D settings for each source file (heh, no
"project-wide" settings here) from a "settings file" that I create outside of
eclipse (by scraping logs). Since we have a dozen different compilers and
several different build processes, this seemed to be the easiest approach.
It took a lot of trial and error to get it to work.
One thing I found with having potentially 1000s of
files in a project each with it's own -I/-D settings (this is mostly when
we use CDT for browsing), is that I had to do something about the size of the
settings file. Since individual -I paths and -D macros are repeated
frequently, I used string pools for those in my settings file and that worked
wonders (at the cost of making my settings file somewhat illegible). I
could be wrong, but I think the equivalent XML file in the CDT .metadata would
be enormous.
Steve Kennedy
From:
cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of
Andrew Gvozdev
Sent: Wednesday, April 28, 2010 12:06
AM
To: CDT General developers list.
Subject: Re: [cdt-dev]
Build Again
There is another special case. Some users do not run builds but use
CDT purely as editing tool. They still need to provide include paths and macro
definitions to the indexer. In clean architecture it should be possible to do
without MBS overhead and even without any build plugins. That's why I suggest
LanguageSettingsContributor extension point in cdt.core (see http://wiki.eclipse.org/CDT/ScannerDiscovery61/Design).
Currently MBS (its part in cdt.core) took over all other
providers/contributors via wrappers and is a single provider of this kind of
information. We know the drawbacks all too well. Instead, I suggest a simple
extension point in cdt.core for any build plugin or another provider. MBS
would be one, new Doug's build plugin would be another.
This does not
negate the need to setup a build toolchain (I take we all agree that scanner
discovery belongs to a toolchain) but it would give a flexibility/ability to
change/add providers at will.
Andrew
On Tue, Apr 27, 2010 at 3:09 PM, Doug Schaefer
<cdtdoug@xxxxxxxxx> wrote:
To Leo's point, have you ever tried to integrate a toolchain where you
don't want MBS, but you do want standard build? The overhead in doing that
and getting (or not getting which is the norm) a working scanner discovery
is what brought me to this point. There is some overlap, but it is actually
quite small. For the android toolchain, it's pretty much the PATH env var
and the names of the compilers and a hook to the gcc scanner discovery.
Unfortunately, I feel we've lost the ability to keep the simple things
simple.
It's not clear to me how managed build calls the built-in
discovery stuff. That's probably all part of my confusion with scanner
discovery to begin with. But as I mentioned, I have no intentions of
changing managed build. My suggestion is to reimplement a whole new build
stack for standard build including a clone/reimplementation of scanner
discovery, and a very simple build model.
On Tue, Apr 27, 2010 at 2:27 PM, Chris Recoskie
<recoskie@xxxxxxxxxx> wrote:
In our case, the only way we can get at the built-ins for the IBM
compiler is to do a hidden build, similar to what is done for the GNU
compilers. There's no Windows registry to rely upon on AIX or Linux that
we could look up such things in. So we need to have the built-ins
discovery run for managed build projects too. I'm not sure how you'd get
around it.
We (IBM) have several compiler versions and languages
that we support, on multiple platforms, and it would be an ugly problem of
combinatoric complexity if we had to hardcode all the different
combinations in the toolchain definitions, or even code it all in Java.
It's even worse for the GNU tools.
===========================
Chris Recoskie
Team Lead,
IBM CDT and RDT
IBM Toronto
Doug Schaefer ---04/27/2010 12:55:59 PM---Thanks, Chris, I
think we're getting a lot of momentum and resourcing to address this. I
can't wait
Thanks, Chris, I think we're getting a lot of
momentum and resourcing to address this. I can't wait to see what we can
do.I'd like to make sure we separate out
scanner discovery from managed build. These are two separate areas that
shouldn't overlap. Again my focus will be on resurrecting a separate
standard build system where scanner discovery is an essential mechanism.
Reusing it for managed build doesn't make sense to me and will probably
simplify managed build if we remove it. For managed build, all you really
want is get at the compiler built-ins and there are probably simpler
mechanisms to accomplish that.:DOn Tue, Apr 27, 2010 at 11:50 AM,
James Blackburn <jamesblackburn@xxxxxxxxx>
wrote:
Hi Chris,
I agree with your summary. I
(thankfully) have got rid of scanner discovery here. The major
gripe here is the issue of merging & version controlling
.cproject. The XML model bears little resemblance to a user's
intuitive understanding of the settings set. I've got a .cproject pretty
printer which generates a human readable output from the config, but the
fact that the .cproject changes seemingly randomly and non-obviously,
after an innocuous change (e.g. flipping active configuration) is
bad.
I'd like to help out improving this and the scalability
issues. I've got some ideas on that front.
Cheers,
James
On 27 April 2010 16:31, Chris
Recoskie <recoskie@xxxxxxxxxx>
wrote:
I would have replied to this thread sooner
but I was on vacation yesterday.
Rather than respond to all the
different sub-threads, I'll summarize a few points here.
As I
stated recently, I think I can get a significant amount of my time put
towards build system work starting in the summer/fall, with the idea
being to release whatever changes resulted in the June 2011 simultaneous
release. Whether everything I would like to do can actually be
accomplished in that time or not remains to be seen, but it should allow
for a good start.
To give some background, we've just GA'ed our
first product release based on our CDT/RDT remote development work...
Rational Developer for Power Systems Software. Basically it's a remote
development IDE for the Power platform. Some issues have come from our
customers as a result during our beta program, most of which surrounded
scalability, build, and scanner discovery. As such we've got the
required momentum behind us to make some investment in these areas if it
will address these pain points.
As others have stated, doing a
complete rewrite of everything would be ideal in some ways but I'm not
sure it's practical either given the resources. That being said. we can
probably rewrite a few things, and in fact in some cases I think we'll
need to if we want them fixed.
So, here are my main concerns and
thoughts. A lot has been said by the rest of you that I would say I
largely agree with... I won't rehash it all, I'll just add to
it.
- Scalability of the build model is a big problem right now.
We already had customers complaining that they have hundreds of projects
in the workspace, which blows up the heap easily. Not only do we need to
address the memory footprint, but the slowness of creating and
persisting the C project description (and particularly the build model
portion of it) needs to be addressed. This slowness gets even slower
when you are working with remote projects, because remote I/O is
expensive. I think the C project description implementation and the
persistence mechanisms need a rewrite.
- The elements in the
current build model/MBS model themselves are not necessarily bad as they
do a pretty good job of modeling arbitrary toolchain setups. Those
elements can probably be tweaked to help to help address the use cases
where we are still lacking, without throwing the baby out with the
bathwater. I think even if I were to think about redesigning the build
model, I'd probably end up in a similar spot to where it ended up, with
a few sins corrected. So I don't think it's worth it to blow that all
away and start fresh. Most of our ISVs integrate to the build system
using the MBS extension points, so keeping that similar is something I'd
consider necessary.
- Autotools support. Some of our bigger
customers are using Autotools with the IBM compilers. Building with them
using a standard make project is really clunky, and scanner discovery
doesn't handle the autotools output properly. I need to look at what the
Redhat folks/Linux Tools folks have done to see how that currently
integrates. Does the autotools integration work with any compiler right
now or just GNU? Did they solve the scanner discovery problem? I don't
know.
- Scanner discovery in general is a complete mess and
personally I would like to raze it to the ground and salt the earth upon
which it stood so nothing ever grows there again. I got scanner
discovery "sort of" working for our product but it is rather hacky and
doesn't work that great. Per file and per folder settings don't
currently work at all, and our customers care about that (see:
Autotools). The scalability of it sucks as the hidden build to get at
the compiler built-ins still gets invoked way too often for arcane
reasons I have not fully deciphered yet. Part of the problem is the
scanner discovery stuff is entangled in the mess that is the
CProjectDescription and its henchmen, so even if you sort of figure out
your scanner discovery parsers and info collectors, getting the data
into the IDE properly is a nightmare.
- Scanner discovery working
properly is REALLY important for us. Our customers have all told us that
our big differentiator that makes our product attractive compared to
vi/Emacs and the like is that we have all these great source-aware
features driven by the index. Our customers have really big projects
though, with lots of third party libraries, and they are pretty much
unwilling to configure the scanner info manually. The big customers
pretty much all have existing build processes that they are unwilling to
switch away from, which means none of them will be switching to Managed
Build when our product supports it.. So scanner discovery has to work
right.
===========================
Chris Recoskie
Team
Lead, IBM CDT and RDT
IBM Toronto
Doug Schaefer ---04/24/2010 11:06:29 AM---Hey gang, FYI, As I
mentioned on the last call, I've been asked to play a more active
role
Hey gang,
FYI,
As I mentioned on
the last call, I've been asked to play a more active role in TCF and
help build a community for it and work with you to bring it to maturity.
That's taking most of my time these days as I get up to speed and figure
out what we need to do with it.
In my spare time, I'm working on bringing the CDT to the
Android community to help build native libraries for that platform. The
focus is on one of my dream projects, building a game engine out of open
source parts, in this case for Android, but I have eyes on iPhone and
MeeGo too. Part of doing that is getting the Android gnu toolchain
working well with CDT. And that's brought me back into the CDT build
nightmare. Once again, I am having a hell of a time getting scanner
discovery working for these seemingly simple Makefile projects. Just
look at a stack trace sometime and you'll see, today, the CDT build
system is the opposite of simple.
So I am urging myself to get back into working on a new build
system. We need to put the architecture back to the way it was, with
managed build being an add-on, not having everything driven by managed
build. Obviously, I'm not going to have much time to spend on it, so by
definition, I'll have to keep it simple. And I'll have to do it as a
separate stack, not as an evolution of what we have.
Over the next few days, I'll jot down
some more ideas here http://wiki.eclipse.org/CDT/NextGenBuild. And I'll recreate cdt.build.* plug-ins, this time in a separate
folder to avoid confusion. And as always help is appreciated and
hopefully, I'll have new found vested interest in bringing it to
completion, especially since I want to get back working on the Android
project.
Cheers,
Doug_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev
mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev
mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev
mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev
mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
