On Sat, Nov 1, 2014 at 1:26 PM, Marco Trudel <marco.trudel@xxxxxxxxxxxx
<mailto:marco.trudel@xxxxxxxxxxxx>> wrote:
On 31.10.2014 20:10, Krishna Narasimhan wrote:
Okay,
Let me tell where your tool would be useful.
* Ability to work on rewritten ASTs. How much code analysis is there
using Codan etc, CDT doesnt give access to intermediate ASTs. The
Refactoring framework handles the final modifications Lets say I
have an
AST that I need to modify in phases. I cant do that unless I write
Refcatorings for each phase, store them in the files (in projects,
retreive them and work on them again.
With your extension, I could just rewrite on the AST, get the
modifed
AST, perform rewrites on them again.
We just have an AST and keep modifying it (you can do this in
Eclipse without any of our extensions). When we're done, we pretty
print the AST to regular files. We don't need to store/retrieve
anything since we don't use an Eclipse workspace/project.
But if I understood right, you're working in the Eclipse IDE. This
means that you somehow have to get your updated AST back into the
model/files/project/workspace/__whatever. I don't know how Eclipse
handles this. You have to figure out if this is possible.
Cheers!
Marco
If I am not wrong.
On Fri, Oct 31, 2014 at 6:23 PM, Marco Trudel
<marco.trudel@xxxxxxxxxxxx <mailto:marco.trudel@xxxxxxxxxxxx>
<mailto:marco.trudel@__mtsystems.ch
<mailto:marco.trudel@xxxxxxxxxxxx>>> wrote:
On 31.10.2014 16:20, Krishna Narasimhan wrote:
This is great.
Is it open source?
Not the translation. That's a commercial product.
Or at least the part where you extended CDT?
Lets see:
> extended multiple AST classes to have more rewriting
support (we
> extended AST rewriting in general).
It would be great if this could be put into CDT. The AST
classes are
sometimes somewhat messy (inconsistent/incomplete). E.g.:
-
IASTFunctionCallExpression.____setArguments(____IASTInitializerClause[])
-
IASTStandardFunctionDeclarator____.addParameterDeclaration(____IASTParameterDeclaration)
The first one is not very nice, but at least it allows you
to add,
remove and insert arguments. The second has a nicer
interface, but
removing and inserting parameters is not possible. You have to
replace the declarator in its parent which might be a function
definition, a standard declaration or another declarator
:-/ And of
course you loose its file location information when
replacing the
function declarator with a new one.
There are more cases like this. We added a rewriter class
that takes
care of all that. But if the interfaces were unified and
extended,
it would be much easier to do AST rewriting in CDT.
> - reimplemented comment handling.
> - reimplemented binding handling.
> - reimplemented type computation.
> - reimplemented constant expression evaluation.
> - reimplemented pretty printing.
This functionality is already in CDT. It's just not always
completely correct, so we had to reimplement it. What's in
CDT now
should be ok for most use cases. Replacing them with our
implementation would break too much of the API.
> - added CFG computation/analysis features.
I assume this can be done with codan. But I didn't find too
much
documentation and judging by what we already had to
reimplement, it
seemed easier to just write that ourselves from scratch.
Especially
since it also has to handle Java AST (e.g. exceptions).
> - added macro analysis/transformation features.
> - added support for keeping user formatting (e.g. empty
lines).
Probably only needed by us and certainly not compatible
with the CDT
API.
I now quickly looked through our CDT patches. We also
rewrote the
assembler handling. This would also be great to get into CDT.
So, except the AST rewriting and assembler support, I don't
think
adding our extensions to CDT makes sense (or is possible).
But I'm
more than happy to help out with getting the other two in.
Thanks!
Marco
On Fri, Oct 31, 2014 at 4:15 PM, Marco Trudel
<marco.trudel@xxxxxxxxxxxx
<mailto:marco.trudel@xxxxxxxxxxxx>
<mailto:marco.trudel@__mtsystems.ch
<mailto:marco.trudel@xxxxxxxxxxxx>>
<mailto:marco.trudel@
<mailto:marco.trudel@>__mtsyste__ms.ch <http://mtsystems.ch>
<mailto:marco.trudel@__mtsystems.ch
<mailto:marco.trudel@xxxxxxxxxxxx>>>> wrote:
On 30.10.2014 01:59, Alena Laskavaia wrote:
Along same lines as Dmitry mentioned, it is
theoretically
possible to
use headless CDT for code transformations. You
do have
to use
workspace
concept,
We use CDT for extensive code analysis and
transformation.
Without
creating a workspace, just by parsing and
processing C files. I
doing this is quite common, but from this thread
it seems
it's not.
This is surprising since we found CDT very useful
for that.
And it
is (IMHO) certainly more maintainable than writing
C or C++
for Clang.
Since what we're doing might be interesting to the
list,
let me give
a quick overview and then CDT related technical
details:
We implemented an automatic C source code to Java
source code
translator:
http://www.mtsystems.ch (contains an online demo and translated
programs)
Such a translation allows to:
- Easily upgrade legacy C projects to Java.
- Automatically create interfaces of C libraries.
- Offer services for Java (e.g. code
analysis/proofs) also
for C.
The translation is completely automatic, supports
the entire C
language and creates functionally equivalent Java
code -
ready to be
executed. Features:
- Pointers are optimized away or translated to
readable Java
classes, function pointers are translated to
method references.
- Full support for unsigned types.
- Full support for goto statements.
- Full support for native libraries; the
translated Java
project
seamlessly interfaces with libraries (GMP,
Ncurses, X11,
...) the C
project used.
- Macros are translated to Java methods or constants.
- Comments are preserved and reformatted to Javadoc.
- The translated code adheres to the Java design
and naming
conventions
- ...
We have evaluated our translation software on a
number of C
programs, including wget, less, xeyes, and micro
httpd. We keep
extending and optimizing it and will soon also put the
translation
of vim online.
As great and helpful CDT was for this project,
there were some
things we had to extend, replace or add. We:
- extended multiple AST classes to have more rewriting
support (we
extended AST rewriting in general).
- reimplemented comment handling.
- reimplemented binding handling.
- reimplemented type computation.
- reimplemented constant expression evaluation.
- reimplemented pretty printing.
- added CFG computation/analysis features.
- added macro analysis/transformation features.
- added support for keeping user formatting (e.g.
empty lines).
And probably some more I forgot now.
Cheers!
Marco
but I don't think it is big deal, because
nobody has to
know
how you doing it internally. I.e. you can
import code
in temp
workspace
as project, configure scanner discovery, run
indexer,
do code
transformations, rewrite code, exit and delete
workspace afterwards.
Location of the code does not have to be
physically inside
workspace, it
is eclipse place to store writeable stuff. And
if you
do all of
this and
it works, would be nice if you contribute this
framework/example
back :)
On Wed, Oct 29, 2014 at 12:02 PM, Dmitry Petrov
<dmitrynpetrov@xxxxxxxxx
<mailto:dmitrynpetrov@xxxxxxxxx>
<mailto:dmitrynpetrov@gmail.__com
<mailto:dmitrynpetrov@xxxxxxxxx>>
<mailto:dmitrynpetrov@gmail.
<mailto:dmitrynpetrov@gmail.>____com
<mailto:dmitrynpetrov@gmail.__com <mailto:dmitrynpetrov@xxxxxxxxx>>>
<mailto:dmitrynpetrov@gmail
<mailto:dmitrynpetrov@gmail>.
<mailto:dmitrynpetrov@gmail
<mailto:dmitrynpetrov@gmail>.>______com
<mailto:dmitrynpetrov@gmail.
<mailto:dmitrynpetrov@gmail.>____com
<mailto:dmitrynpetrov@gmail.__com
<mailto:dmitrynpetrov@xxxxxxxxx>>>>> wrote:
You can implement a stand-alone
application based
on CDT.
org.eclipse.cdt.codan.______internal.core.CodanApplication
might be a good
example if you want to do something with
source code.
Let me share some of my experience as a
person who
uses
both Clang
and CDT for tool development.
Unfortunately, neither Clang nor CDT
solves C/C++
source-to-source
transformation issues related to the
preprocessor
quite
well. That's
a very difficult area, though.
ClangTooling doesn't add anything special
to Clang
ASTs model.
Clang doesn't represent preprocessor
statements in
AST.
There are
some mechanisms to match comments to
declarations, but
nothing more.
You'll have to hack things on top of
preprocessor
callbacks
and AST
traversal yourself.
On Wed, Oct 29, 2014 at 6:49 PM, Krishna
Narasimhan
<krishna.nm86@xxxxxxxxx
<mailto:krishna.nm86@xxxxxxxxx>
<mailto:krishna.nm86@xxxxxxxxx
<mailto:krishna.nm86@xxxxxxxxx>__>
<mailto:krishna.nm86@xxxxxxxxx <mailto:krishna.nm86@xxxxxxxxx>
<mailto:krishna.nm86@xxxxxxxxx
<mailto:krishna.nm86@xxxxxxxxx>__>__>
<mailto:krishna.nm86@xxxxxxxxx
<mailto:krishna.nm86@xxxxxxxxx>
<mailto:krishna.nm86@xxxxxxxxx
<mailto:krishna.nm86@xxxxxxxxx>__>
<mailto:krishna.nm86@xxxxxxxxx
<mailto:krishna.nm86@xxxxxxxxx>
<mailto:krishna.nm86@xxxxxxxxx
<mailto:krishna.nm86@xxxxxxxxx>__>__>__>> wrote:
I tried clang. Not the ClangTooling.
The core
Clang
seemed to
have issues like not preserving
comments after
rewrites
etc. Do
the Tooling handle Preprocessor
statements too?
I thought clang was primarily an
optimmization
framework and not
a source code transformation tool.
On Wed, Oct 29, 2014 at 4:40 PM,
Nathan Ridge
<zeratul976@xxxxxxxxxxx
<mailto:zeratul976@xxxxxxxxxxx>
<mailto:zeratul976@xxxxxxxxxxx
<mailto:zeratul976@xxxxxxxxxxx>__>
<mailto:zeratul976@xxxxxxxxxxx <mailto:zeratul976@xxxxxxxxxxx>
<mailto:zeratul976@xxxxxxxxxxx
<mailto:zeratul976@xxxxxxxxxxx>__>__>
<mailto:zeratul976@xxxxxxxxxxx
<mailto:zeratul976@xxxxxxxxxxx>
<mailto:zeratul976@xxxxxxxxxxx
<mailto:zeratul976@xxxxxxxxxxx>__>
<mailto:zeratul976@xxxxxxxxxxx
<mailto:zeratul976@xxxxxxxxxxx>
<mailto:zeratul976@xxxxxxxxxxx
<mailto:zeratul976@xxxxxxxxxxx>__>__>__>> wrote:
> Basically I would like to
perform write test
codes that work on ASTs
> not obtained from any project
from the IDE.
Basically Parsed from
> Strings or other text files.
Perform
modifications to that AST and be
> able to use the rewritten AST ,
all this
possibly
without a code from
> the IDE
If you're writing a standalone
tool for
creating,
analyzing,
and rewriting
ASTs, you might want to consider
the clang
infrastructure [1].
Regards,
Nate
[1]
http://clang.llvm.org/docs/______Tooling.html
<http://clang.llvm.org/docs/____Tooling.html>
<http://clang.llvm.org/docs/____Tooling.html
<http://clang.llvm.org/docs/__Tooling.html>>
<http://clang.llvm.org/docs/____Tooling.html
<http://clang.llvm.org/docs/__Tooling.html>
<http://clang.llvm.org/docs/__Tooling.html
<http://clang.llvm.org/docs/Tooling.html>>>
_____________________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>
<mailto:cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>>
<mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx> <mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx>>>
<mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx>
<mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx>> <mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx>
<mailto:cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>>>>
To change your delivery options,
retrieve your
password, or
unsubscribe from this list, visit
https://dev.eclipse.org/______mailman/listinfo/cdt-dev
<https://dev.eclipse.org/____mailman/listinfo/cdt-dev>
<https://dev.eclipse.org/____mailman/listinfo/cdt-dev
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev>>
<https://dev.eclipse.org/____mailman/listinfo/cdt-dev
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev>
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev
<https://dev.eclipse.org/mailman/listinfo/cdt-dev>>>
--
------------------------------______-----------------------
I dare do all that may become a man;
Who dares
do more,
is none
- Macbeth, twelfh night!
Regards
Krishna
_____________________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>
<mailto:cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>>
<mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx> <mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx>>>
<mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx>
<mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx>> <mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx>
<mailto:cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>>>>
To change your delivery options,
retrieve your
password, or
unsubscribe from this list, visit
https://dev.eclipse.org/______mailman/listinfo/cdt-dev
<https://dev.eclipse.org/____mailman/listinfo/cdt-dev>
<https://dev.eclipse.org/____mailman/listinfo/cdt-dev
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev>>
<https://dev.eclipse.org/____mailman/listinfo/cdt-dev
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev>
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev
<https://dev.eclipse.org/mailman/listinfo/cdt-dev>>>
_____________________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>
<mailto:cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>>
<mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx> <mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx>>>
<mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx>
<mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx>> <mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx>
<mailto:cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>>>>
To change your delivery options, retrieve
your
password, or
unsubscribe from this list, visit
https://dev.eclipse.org/______mailman/listinfo/cdt-dev
<https://dev.eclipse.org/____mailman/listinfo/cdt-dev>
<https://dev.eclipse.org/____mailman/listinfo/cdt-dev
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev>>
<https://dev.eclipse.org/____mailman/listinfo/cdt-dev
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev>
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev
<https://dev.eclipse.org/mailman/listinfo/cdt-dev>>>
_____________________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>
<mailto:cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>>
<mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx> <mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx>>>
To change your delivery options, retrieve your
password, or
unsubscribe from this list, visit
https://dev.eclipse.org/______mailman/listinfo/cdt-dev
<https://dev.eclipse.org/____mailman/listinfo/cdt-dev>
<https://dev.eclipse.org/____mailman/listinfo/cdt-dev
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev>>
<https://dev.eclipse.org/____mailman/listinfo/cdt-dev
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev>
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev
<https://dev.eclipse.org/mailman/listinfo/cdt-dev>>>
_____________________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>
<mailto:cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>>
<mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx> <mailto:cdt-dev@xxxxxxxxxxx
<mailto:cdt-dev@xxxxxxxxxxx>>>
To change your delivery options, retrieve your
password, or
unsubscribe from this list, visit
https://dev.eclipse.org/______mailman/listinfo/cdt-dev
<https://dev.eclipse.org/____mailman/listinfo/cdt-dev>
<https://dev.eclipse.org/____mailman/listinfo/cdt-dev
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev>>
<https://dev.eclipse.org/____mailman/listinfo/cdt-dev
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev>
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev
<https://dev.eclipse.org/mailman/listinfo/cdt-dev>>>
--
------------------------------____-----------------------
I dare do all that may become a man; Who dares do more,
is none -
Macbeth, twelfh night!
Regards
Krishna
___________________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>
<mailto:cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>>
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/____mailman/listinfo/cdt-dev
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev>
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev
<https://dev.eclipse.org/mailman/listinfo/cdt-dev>>
___________________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>
<mailto:cdt-dev@xxxxxxxxxxx <mailto:cdt-dev@xxxxxxxxxxx>>
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/____mailman/listinfo/cdt-dev
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev>
<https://dev.eclipse.org/__mailman/listinfo/cdt-dev
<https://dev.eclipse.org/mailman/listinfo/cdt-dev>>
--
------------------------------__-----------------------
I dare do all that may become a man; Who dares do more, is none -
Macbeth, twelfh night!
Regards
Krishna
--
-----------------------------------------------------
I dare do all that may become a man; Who dares do more, is none -
Macbeth, twelfh night!
Regards
Krishna
--
-----------------------------------------------------
I dare do all that may become a man; Who dares do more, is none -
Macbeth, twelfh night!
Regards
Krishna
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cdt-dev