Eclipse Community Forums: B3 » Ideas on b3 "build files" using simple xtext based syntax...

Help

Home

Home » Archived » B3 » Ideas on b3 "build files" using simple xtext based syntax...

Show: Today's Messages :: Show Polls :: Message Navigator

Ideas on b3 "build files" using simple xtext based syntax... [message #489423]

Fri, 02 October 2009 22:45

Henrik Lindberg

Messages: 2509
Registered: July 2009

Senior Member

Hi,
I have been working with some ideas what an xtext based build file for
b3 could look like - how things like resolver configuration, and advice
can be expressed in a simple and compact way.

Below are some thoughts on what it could look like... (the ideas here
reflect what is available in the models I posted earlier).

Perhaps I got a bit carried away, but I am excited about the possibilities.

So - rip it apart please... ideas, questions, critique...

XText Syntax Ideas for b3 build files (version 0.1)

This document contains ideas for an xtext based syntax describing build
meta data. I have taken liberties with the syntax, you have to read it
with a "do what I mean" attitude, as this proposal otherwise may be very
difficult to handle for a parser. This first draft is just to vent some
ideas, not to nail down the exakt syntax.

As started to elaborate on some of things you may want to do in Xtext, I
found that I needed to describe some of the design ideas captured in the
models, so this text also includes some explanations of the b3 models.

The idea is that a b3 build file is used as input to b3 commands.
Commands in b3 are in practice just two basic commands - either
"import" (to populate the workspace), or "invoke" (to get the value (or
side effect) of a build unit part (e.g. MyProduct#createProductZip).
(Although, there may be other commands that can be issued that b3
handle, like querying, debugging, dumping, logging, etc. - but in
principle, there is just "import" and "invoke").

It is possible to do both import and return a result at the same time.
The only difference is that the "import" does not require specification
of a particular part, and that the typical use of "invoke" is to perform
it against what has already been imported to (or preexisted in) the
workspace, thus not requiring a resolver specification other than the
default (workspace and target platform).

CONTEXT
The b3 build file can be thought of as a configuration of the context in
which commands to b3 execute. The b3 context contains:
- units (the build units in the context)
- resolutions (the resolution of each required capability)
- advice (the set of advice to apply in the context)
- requests (the input request(s), and requests originating from required
capabilities)
- resolver (the network of resolvers to use to resolve
requests/requirements).

Each request sets up a new context (it may inherit from the default
context), and it is (optionally) configured with a build file.

RESOLVER
A context has a resolver network consisting of resolver nodes. Two
special nodes "first" and "best" are used to specify how the search is
conducted - resolvers in a "first" list results in the first found
resolution, and resolvers in a "best" list results in the resolution
with the highest score (compared to the request).

A b3 build file makes it possible to specify the resolution network with
a simple syntax.

The specified resolution node itself is a "first" type of node.

import org.eclipse.b3.resolvers.*; // import names of resolvers to
reduce typing

resolution {
TargetResolver;
WorkspaceResolver;
}

This specifies that an attempt is made to resolve against the target
platform, and if that fails against the Workspace. No options are
specified for the resolvers.

What if I want to resolver first from Target platform, but not for
certain tool bundles that, if I have them in the workspace, I would like
to get those instead of the installed. This could look something like this:

resolution {
WorkspaceResolver {
namePattern="org.myorg.mytool.*";
};
TargetResolver;
WorkspaceResolver;
}

Here is an example using best:

resolution {
TargetPlatformResolver;
WorkspaceResolver;
Best {
P2Resolver {
location = "http://www.someplace.good";
}
SVNResolver {
location = "svn+ssh://someplace.else";
...;
};
}
}

There are several settable properties in the different resolvers.
(Compare to Buckminster's Readers in the RMAP).

Additional features (TBD):
Here are some additional things I would like to be able to do:
- name top level resolution specifications
- reuse a named resolution in another resolution
- import resolutions from another build file, and reuse a named
resolution in the imported build file
- use of properties

Possibly also:
- specify an import with a translator to interpret the file in a
particular location

ADVICE

// advice sets values in the model
// similar to CSS
// An XPath is a query, it matches a set of nodes, where a value can be set
//
advice {
xpath = value;
...
xpath {
rel-path = value;
...
}

The xpath is relative to the context - which is analogous to the
"document" in XML/HTML.
At the root of the context there are:
- units (the build units in the context)
- resolutions (the resolution of each required capability)
- advice (the set of advice to apply in the context)
- requests (the input request(s), and requests originating from required
capabilities)
- resolver (the network of resolvers to use to resolve
requests/requirements).

It is unclear if it should be allowed to apply advice on all of these
(i.e. advice on advice). The idea is to primarily advice on requests,
and resolutions, and in some cases on units.

So here are some examples:
// All requests to org.myorg.projx.* should be requests for source
/requests[name=org.myorg.projx.*]/options/source = true;
// All requests to org.myorg.projx.* should be requests for mutable
/requests[name=org.myorg.projx.*]/options/mutable = true;

// shorthand for the two pieces of advice above
/requests[name=org.myorg.projx.*]/options {
source = true;
mutable = true;
}

// if I only need mutable source for the eclipse.features
/requests[name=org.myorg.projx.*, namespace="eclipse.feature"]/options/ {
source = true;
mutable = true;
}

Options include: source, mutable, branchTagPath, timestamp, revision,
resolverFilter, filterGroups, overlayPath, includeParts, excludeParts,
and prune.

Advice on resolutions
Here are some examples

// Place all of my organization's "core" components in the target platform
// (the rest goes to workspace by default).

/resolutions[name=org.myorg.core.*] /options {
materializer = org.eclipse.b3.targetPlatformMaterializer;
location = /where/it/should/be/stored;
conflictResolution = update;
}

It is possible to make very advanced advice:

/resolutions/[name=org.myorg.*]/subResolutions/resolution[na mespace= "org.myorg.annotations"]/options
{
materializer = org.eclipse.b3.workspaceMaterialzer;
location = /where/they/are/stored;
}

This would store all build units that provide a capability in the
namespace org.myorg.annotations in a particular location on disk and
link them to the workspace.

Advice on units
The advice on units is applied as units are resolved. This means that
the advice has an effect during the contexts lifetime, but as soon as
the operation is over, the live model will reflect the original meta
data translation.

The same advice can be applied again in some other context if needed.
Sometimes all that is required is to override some faulty meta data
(that the user wants to correct) to make resolution possible. As an
example, we want to override a components request for a particular
required capability. Once the components has been resolved it will
probably not build, the problem is corrected by the user and the unit
checked in. The next time, the overriding advice is not needed.

/units[name="org.myorg.projx.broken"]/requiredCapabilities[name= "org.myorg.x"]/versionRange
= [1.0.0, 3.0.0];

Creation of complex types.
Most of the time values are simply strings, or numbers, or there is a
simple translation from string form to the datatype required for a
settable attribute. Sometimes this is not possible, and a "new instance
operator" is required. This could look something like

path-expr = new RequiredCapability {
name = ...;
nameSpace = ...;
...
};

Synthetic nodes
It must be possible to address synthetic nodes in the document such as
the "before first" and "after last" positions in a list. This is done by
suffixing the xpath expression iwth ":synthetic node name"

e.g. /units[name=org.myorg.foo]/requiredCapabilities:first = new
RequiredCapability { ... }.

Lists
It must be possible to reference the list itself. Clear the list. Remove
an entry in the list, add an element first or last, and before or after
an element. (Have to look more at XPath to see how this is typically
done to give examples).

UNITS
As the advice mechanism is very powerful it is actually possible to
create build units directly in the b3 build file. This is useful in some
cases where a virtual/configuration unit is required to define the order
of dependencies. In many cases, creating an additional bundle or feature
that is only used for some special aspect of building/testing can be an
overkill - so it is quite convenient to declare it directly in the b3
build file.

Apart from this, the build units are mostly the result of an import, and
they may be advised (as described earlier).

REQUESTS
Requests, as you have already seen are the original "import statements"
and requests made by "required capabilities" in the resolution being
performed (as part of an import or an invocation). The requests can be
advised, but are otherwise not terribly interesting :). Note that it is
possible ro request something in any namespace, but that the resolution
will always resolve this into some build unit providing the capability
(analogous to how p2 does this).

RESOLUTIONS
Resolutions are the result of resolving - they tie the resulting build
units to the requirements that caused them to appear in the context.
Interesting, advanced things can be done with this as shown in the
advise section, but mainly this is a mechanism to tie everything
together and enable the invocation of actions that traverse the
resulting model.

(I probably forgot to describe something that is important... but this
post is already long enough as it is...)

- henrik

Breadcrumbs

Sign up to our Newsletter