Re: [equinox-dev] Good support for PDE in build systems anyone?

[Jeff McAffer <Jeff_McAffer@xxxxxxxxxx>]

Peter Neubauer wrote on 06/04/2006 03:47:51 PM:

> in particular I find the use of Maven plugins and other tools not
 
> sufficient when it comes to developing OSGi bundles in the Eclipse
 
> PDE. The problem is, compile time everything is fine having  
> dynamically mounted jars via virtual classpath containers a la  
> MevenIDE, and the old Magic plugin I wrote. But even so, PDE requires
 
> a valid Manifest to be present under /meta-inf of the current OSGi
 
> bundle project in the workspace, and the PDE dependencies virtual
 
> classpath variable mounts the deps mentioned in the Manifest.
>
> However, when you try to run/debug your plugin,
the dependencies  
> stated in the Classpath entry in the manifest have to be there under
 
> the bundle root folder.

It is conceivable that the manifest be somewhere else.
 Not sure what exactly the issue is here.  That is, if you have
a manifest in your project in your workspace, is it an aesthetic thing
that would cause you to put is somewhere other than in META-INF?  or
is there a technical issue?  As you note, this is not just PDE being
anal.  When you turn around and want to run your bundles, we like
to avoid any sort of build/deploy/...  so if the manifest is going
to be somewhere else, the framework you are running still has to find it/be
told about it.

Metanote:  the whole 1:1 project:bundle mapping
issue revolves around our desire to present at compile time, a classpath
that is as close as possible to the classpath that will be used at runtime.
 Since we build a project at a time and each project has one classpath,
you get the current behaviour.  To date the value of having an accurate
classpath at build time has outweighed any issues wrt the mapping.

> Basically, the problem is that the PDE is taking
the Manifest and  
> OSGi as master, where maven and others are generating everything from
 
> a POM/Ivy file. 

Yes, this is a characteristic that Peter Kriens has
been pointing out.  Certainly there are cases where you need metadata
at development time that is not needed at runtime.  Having an additional
file to hold this information is a good idea.  PDE uses build.properties.
 Somewhat crude but functional.  

The point about wanting to run without build/deploy
is relevant here too.  PDE has several mechanisms that work to keep
the manifest "runnable" at all times.  They also feed back
error/warnings as you make the problems so there are fewer surprises.  

Increasingly people are using the PDE graphical editors
(as opposed to just the source page) so in a sense the underlying format
of the data and what file it is in is decreasingly important to PDE.  (of
course Wassim and the folks who actually write that code may have a different
opinion :-)  However, it is not clear that coming up with N different
syntaxes to represent what is already so well set out in the spec makes
sense.  The maven-osgi plugin has its own way of spec'ing the dependencies.
 Not sure about Ivy.  Why not just use the manifest?

There was some very initial discussion with some Maven
folks about the possibility of having Maven read manifests directly and
use them to populate the internal model.  Nothing has happened there
AFAIK.

> In Ivy, we have solved that somewhat by
> - reading most of the attributes from the Manifest.mf as master  
> instead of the ivy.xml
> - download the different deps locally before compiling and using them
 
> for the classpath from within the PDE. Generated by Ivy but still
in  
> harmony with the PDE we have a working build system with Ivy that
 
> works good with the PDE at https://scm.ops4j.org/repos/ops4j/ 
> laboratory/users/andreas/builder/
> - package the bundle with a similar structure as the development layout.

Is this working for you?  What are the gotchas?

BTW, what is hte license on Ivy?

> I'm not sure how to solve this discrepancy between
the two mindsets
> - runtime, OSGi rules and the PDE tries to enforce valid development
 
> time editable artifacts
> - metadata rules and all runtime data is generated from POM/Ivy  
> metadata, which is the build system approach.
> 
> I see two major points where the PDE has to be adapted to honor the
 
> build system or the other way round:
> 
> 1) Manifest file
> - one could think of a Maven builder generating new manifest.mf on
 
> the fly as you edit the relevant POM/other metadata, like source code
 
> triggering changed import statements

Sure.  Note that PDE does not have to do anything
here.  It will pick up any changes made to the manifest by such a
builder.

> - flexible location of the Manifest in the bundle or at least  
> allowance in the PDE to specify a manifest by URL (to be able to  
> reference it as a custom protocol "artifact:mf:my/project/ 
> manifest#1.0.2") pointing to a maven resolved repository or generated
 
> location.

No doubt the implementation of PDE is expecting the
manifest ot be in the /META-INF dir but aside from that, conceptually the
manifest might be somewhere else at development time.  However, at
runtime someone is going to have to either put it in the standard spot
or tell the framework where it is.

> 2) Dependencies
> - not sure that is possible but it would be great to be able to  
> reference everything in the PDE as URLs, opening for mounting e.g.
 
> the runtime jars from the local maven/ivy repo into the right place
 
> for the PDE. Somewhat possible with Eclipse workbench links I guess
 
> but they seem to be tied into absolute file paths and protocols,  
> making them too static for this.

PDE works on the notion of a "target platform".
 This is a set of bundles that are in the filesystem and not in your
workspace.  Think of it like the Maven local repo.  In 3.2 the
target platform can source bundles from several locations and the expected
structure of these locations has been relaxed somewhat.  This is not
enough as the Maven, for exmaple, structure is potentially deep and PDE
does not search.  One also has to consider whether or not you want
the whole local repo exposed as part of the target.  Typically the
target is used to scope down what you are expecting to be available...
 In any event, there has been some work in the direction.  See

        https://bugs.eclipse.org/bugs/show_bug.cgi?id=124542

> - best would be to be able to even in the manifest.mf
bundle  
> classpath to use URLs for the private jars etc. in order to be able
 
> to use the same approach
> - this would enable things like Hansa/Transit that is Niclas favorite
 
> pet ;) http://wiki.ops4j.org/dokuwiki/doku.php?id=hansa:hansa

by "private jars" do you mean the ones on
the classpath of the bundle?

Some of the above issues may actually turn out to
be JDT things in as much as all PDE is really doing is populating a JDT
classpath.  I don't think that they support URLs in any form on the
classpath.  No idea what the technical issues would be there.

> Not sure how much of this sounds understandable
and how much there is  
> support for it in the OSGi spec and Eclipse PDE possibilities, just
 
> thinking aloud.
> 
> WDYT? does anyone out there have a nice working PDE system with Maven
 
> working for OSGi bundle development and a description of the process?

Certainly allowing more people to work in the way
they are accustomed to is a good thing.  

Jeff