(I changed the subject because Jan brings up lots of important stuff that extends far beyond LLVM/clang integration.)
Jan,
Your observations are all very accurate. In particular the lack of sensible/easy instantiation of the build model. This is an area that many people have spoken about recently, now the challenge is how to move forward. I think in the Eclipse Ecosystem such a build model should be represented in EMF models. Alexander Federov and William Riley have both been making some discussions in that regard.
The good makefiles is an interesting point - do other tools such and Meson and CMake make "good makefiles"? At one point Eclipse CDT folk implemented a new builder that (essentially) reimplements Make called the Internal Builder.
What we need is a way forward - and I don't think anyone is excited about evolving as opposed to replacing the existing Managed Make / GNU Makefile Generator code. The current code is very brittle, used by lots and lots of extenders so any change, even small requires a very large amount of work to decide if it is safe. However perhaps I am wrong about this and I should be convinced otherwise.
To answer your questions:
1)Inside the broad CDT world. Does anyone know of other build models like described in the Managed Build System Extensibility Document in the eclipse help->CDT-plugin developer guide->programmers guide->CDT DOM?
I don't know of other build models - in the broad CDT world AFAIK almost everyone either uses MBS or has custom Makefiles or other tools to manage build settings.
2)Did you ever build a build model based on "Managed Build System Extensibility Document" or others? Can you share your experiences/thoughts?
Yes, the build model is what you are using in Sleober :-)
3)Suppose I build a buildmodel instantiator (and a makefile generator for testing) does anyone think someone will use it?
This is an area that I cannot answer. I think having a good instantiation of the build model that is easier to use than what we have today would be nice. But as I outlined above, I think that the development effort may best be spent on a step change to new technologies.
4)Could this buildmodel instantiator become part of CDT?
Yes it absolutely can if it is a testable unit of code. If it is useful to you it will probably be useful to others.
Thanks Jan for re-starting this discussion here. There have been lots of discussions on these issues in the past, on the mailing list and some written down in the Wiki*. What is needed is someone to lead this effort and a real commitment of development resources to make it happen.
* I have been reviewing all the Wiki pages. Some pages that I have looked at on this topic are:
On Sat, 18 Jan 2020 at 08:51, Jan Baeyens <jan@xxxxxxxxxx> wrote:
Hi
I'm happy to there is lots of attention for the managed build
capability of CDT right now :-).
I have been playing with the gnu makefile generator for some years
now and I have been impressed with the capabilities provided by
the extension points (called build model below).
I realize I only understand a very small portion of the problem
domain. For instance I don't know cMake, LLVM, LLVM-GCC, LLVM +
JIT... but I also feel I can contribute.
What I do know is this; Whatever tool you use to build, you will
always have to combine source code files with commands (executable
and parameters).
The way to combine (and the commands) can be very easy or very
complex.
When I look at the gnu makefile generator I think the build model
is a very good basis to describe "how to combine source code files
with commands".
If I look at the implementation of the gnu makefile generator I
see 3 problematic design decisions:
1)No instantiation of the build model.
2)Trying to make "good makefiles"
3)Working with strings
I would like to sell the lack of "instantiation of the build
model" as a current weakness in CDT and the addition of a
"instantiation of the build model" as the solution.
And please see my questions below.
1)No instantiation of the build model.
Suppose I have the super simple build model
convert each *.c to *.o with the command g++ -o {output} {input}
convert all *.o to {project}.exe with the command link {inputs}
{output}
With a project myExample, containing the files input.c and
test.c;
A instantiation of the build model would look like
My thinking is that with this instantiation of the build model
one can easily
1) make a makefile
2) run the commands
3) make a cmake file
....
Even though gnuMakefileGenrator does all this work; it doesn't
instantiate the build model in any way, the code goes from build
model directly to makefile in pieces. If we could agree on a data model for the instantiation of the
build model we could avoid lots of duplication and have a great
addition to CDT.
2)Trying to make "good makefiles"
The second problem I see with gnuMakefileGenerator is that it
tries to generate good makefiles. Hints to this are:
1)The possibility to not expand environment variables in the
makefile
2)When using nameproviders the makefile becomes completely
different
Though it is noble to make readable/maintainable makefiles, it
also requires lots of extra resources (like testing) and a high
level of knowledge about makefiles for future maintainers.
Using only one "type" of makefiles one can easily reduce:
1)amount of code
2)amount of tests
3)amount of training
3)Working with strings
The code is all based around strings and doesn't use classes for
files and so on. I don't think this is due to lack of skill but a
simple consequence of the design decisions I already mentioned.
This however adds to the maintenance nightmare.
My questions
My fingers want to tackle this problem but to avoid needless work
I feel it is important for me to have a better understanding of
how this fits into the cdt community. As such I have following
questions.
1)Inside the broad CDT world. Does anyone know of other build
models like described in the Managed Build
System Extensibility Document in the eclipse help->CDT-plugin
developer guide->programmers guide->CDT DOM?
2)Did you ever build a build model based on "Managed Build
System Extensibility Document" or others? Can you share your
experiences/thoughts?
3)Suppose I build a buildmodel instantiator (and a makefile
generator for testing) does anyone think someone will use it?
4)Could this buildmodel instantiator become part of CDT?
Best regards
Jantje
Op 18/01/2020 om 4:50 schreef
Marc-Andre Laperle:
Hi,
From what I remember, the intention of the preference page was to
be able to add the system headers which a prebuilt clang did not
know about at all and you had to specify manually. This preference
page would be global to all projects and configurations. But this
is not the way normally managed build projects are configured. In
fact, it did this global setting by adding the include paths to
every projects and every configuration, before every build and
regardless of the toolchain! Also, post-build it would always do a
full workspace refresh which I had to comment out because people
were complaining quickly that Eclipse was unusable while this
plugin was installed.
In the midst of the enthusiasm of having LLVM
support, I don’t think this plugin was fully tested or
thoroughly reviewed because there were/are glaring issues. I
personally ended up using the GCC toolchain and just replacing
the command with clang and clang++ which worked fine (especially
once many OS/distros had a prebuilt clang fully configured with
system paths, etc) so the motivation of fixing the LLVM
toolchain was/is low. Still, I think it would be a nice project
to go back and clean up the LLVM plugin (…or remove it). A first
obvious clean-up would be to remove the preference page and any
concept of global include path and libs. Another step would be
to remove the many tools and toolchain that I don’t believe
people use or are still maintained (LLVM-GCC, LLVM + JIT, etc).
Then there is general code clean-up (there shouldn’t be obvious
comment for every line).
In my new role as CDT project lead there
are a number of areas that I am simply not familiar
with (yet!) and this is one of them.
I can try to provide some background, and
I will be very happy to review any patches you
provide!
To answer some of your questions:
The LLVM plug-ins were added in Bug 338553 and
haven't changed much since then in the particular area
of the code you have highlighted.
From a quick look at this it seems there
is a confusion in this code between settings passed to
the LLVM compiler and those passed to the CDT indexer
and which tool is "in charge". In simple cases that
difference probably doesn't matter, but it clearly
quickly goes wrong (the case in Bug 527757 is
particularly obviously bad).
> - Is there any (high level) design
documentation available for the LLVM managedbuilder
stuff? Or for GCC?
Have a look at https://wiki.eclipse.org/CDT/designs/MBS it
contains design documentation for when managed build
was first added and has a number of links that may
be useful.
> - Why is there a CDT LLVM preference
page? What is/was it's purpose?
I don't know. There may be answers in
the Petri's thesis, but the link to it is dead (see comment).
> There is no such page for GCC?
Because it probably shouldn't exist for
LLVM either?
> - I cannot find UI for LLVM/Clang
project properties. Can someone
enlighten me?
> - The same for the GCC project
properties?
It is in the Project Properties ->
C/C++ Build -> Settings. (see attached file to
confirm we are talking about the same thing)
Thank you for taking the time to look at
this. There is certainly a lot of little bits needed
to get a C++ Managed Build project to "just work" and
I look forward to supporting you where I can.
My immediate personal problem is solved by placing a
comment on line 62
of LlvmResourceListener.java to prevent
LlvmPreferenceStore.addStdLibUnix() being called.
I suspect that the same should be done on line 59 but
I have no Windows
with MinGW to test it with.