|filterSingletonCollisions looks like it will help for singleton bundles that we want to allow to be resolved in multiple regions. The difficulty, as you observe, is knowing when this is applicable as some singleton bundles really are framework singletons. So I think it will be necessary to add another property to the user region configuration file to declare the singleton bundles which we want to allow to be resolved in multiple regions.|
I am already planning to add another property to the user region configuration file to specify which bundles are imported from the kernel region into the user region. The use case here is AspectJ, where it is very handy to be able to use import-bundle rather than importing many packages.
These two properties will interact as follows. If a singleton bundle is specified in both these properties and is resolved first in the kernel region, then it will fail to resolve in the user region since the singleton is imported from the kernel region. However, resolving in the opposite order is a little tricky. Suppose the singleton in the user region is resolved first. Then I think it would be necessary to fail the resolution of the singleton in the kernel region even though the singleton bundle is not imported from the user region to the kernel region.
I'll have to give this design more thought as it doesn't fit the digraph structure very well, but I think it probably should.
On 7 Feb 2011, at 08:34, Борислав Капукаранов wrote:
I managed to get my branch starting and activating DS components fine. As a workaround i removed the "singleton" attribute in the DS bundle(org.eclipse.equinox.ds) located on build.eclipse.org
that we build against.
As i expected that solved the issue, but it leaves an important question - What are we doing with the hooks and the singleton bundles? There are certanly many singletons out there so its important that is clear as a bell. I can see singletons are covered to some extent in the spec. E.g. this is a requirement related to the resolver hook:
"Remove specific singleton bundles from influencing the resolvability of another specific singleton bundle."
And here is the technical solution section for the singleton bundles effect:
"5.5.4 Effect of Singleton Bundles
A Resolver Hook can change the effect of singleton bundles using the following method:
• filterSingletonCollisions(Capability singleton, Collection<Capability> collisionCandidates) – An osgi.bundle singleton capability has the same symbolic-name as the given collection of osgi.bundle capability candidates. The implementor of this method can optionally shrink the list of collision candidate capabilities. ..."
It seems this filter can be used to limit/remove the effects of singletons defined by the implementor on the resolving. So i guess it'll have to be handled case by case for each singleton?
I'm curious, Glyn have you used or looked into that method during the implementation of the hooks? Do you think it can help here - i think it is designed to help in situations such as this.
2011/2/4 Борислав Капукаранов <b.kapukaranov@xxxxxxxxx>
Actually there are no bundles using the DS bundles directly. The bundles just provide xml files that declare this bundle has a DS component. When the DS bundles start, simply put, they browse all started bundles for such xmls, described in the manifests and start listening for bundle-started events in order to activate every newcomer as well. That means that DS are not a service that you can export, they are not a list of packages either - in order to work they just have to *see* the bundles in the framework they are in at the moment.
After the DS components are read and processed by the DS bundles an OSGi service is published where needed (e.g. the DS component declared it provides a service).
IMO the DS should be somehow present in both user and the kernel region as they both has DS components that are independent of the other region's.
Basically this is needed mostly for the agent.dm
bundle - it declares two DS components. Since that bundle is present in both regions i think its supposed to have its DS components in both regions not as shared(cross-region?) DS components but as new ones per region.
I imagine this as enabling the installation in both regions, currently installing DS in the user region fails in Quasi resolving stage with the error that:
[2011-02-04 17:05:20.772] OSGi Console <DE0002E> Installation of bundle 'org.eclipse.equinox.ds' version '1.2.100.v20101206' failed. org.eclipse.virgo.kernel.osgi.framework.UnableToSatisfyBundleDependenciesException: Unable to satisfy dependencies of bundle 'org.eclipse.equinox.ds' at version '1.2.100.v20101206': Cannot resolve: org.eclipse.equinox.ds
The bundle could not be resolved because another singleton bundle was selected. In bundle <org.eclipse.equinox.ds_1.2.100.v20101206>
As for the question if any package exports are needed, every bundle that declares it has DS components contains also a class with activate and deactivate methods. This class is specified as the implementation class for the DS component. Usually such classes use activate and deactivate with one argument of type org.osgi.service.component.ComponentContext. So the bundle containing this class imports org.osgi.service.component, which is exported from the DS bundles.
BTW you can sync my dev branch bug333474-DS-remove-startup-order - i've merged it with the framework-hooks branch and experiment on top of it.
I hope this clears the mud a bit :)
2011/2/4 Glyn Normington <gnormington@xxxxxxxxxx>
I'd like to understand the DS dependencies better. I want to avoid the hooks having to special case DS.
Do bundles using DS simply need to wire to certain DS packages or do they need to see anything else (such as DS bundles or, very unlikely, services)?
The more complex issue seems to be DS's need to see the DS components in the user region. What visibility is required? If DS needs to see bundle events, that would cause problems (because Spring DM running in the kernel region would process application contexts in the user region).
For Spring DM we run separate copies in the kernel and user regions so the user can change the version of Spring in the user region, but that seems overkill for DS.
I suspect we may need to put DS in its own region and connect it to/from both the kernel and user region with appropriate filters...
On 4 Feb 2011, at 02:26, Борислав Капукаранов wrote:
I merged bug330706-framework-hooks
into my dev branch and encountered some problems. The kernel starts normally only when lib/kernel/org.eclipse.virgo.kernel.userregionfactory is the last bundle described in the launcher.properties, so this is kind of a new startup dependency. When i placed it around the half of the list of kernel bundles it hanged in STARTING state leaving the kernel frozen. Apart from that there are no other new dependencies, but we'll have to see if there is a solution for userregion.factory.
The user region however fails to start everytime - now there is only one framework but a set of bundles that should be available for both regions (such are the DS bundles) it is kind of problematic when they are singletons. I tried several approaches to workaround the issue, first i moved their declaration from the userregion plan to the kernelshadow plan - at first i though it had some special handling that might help - it works as regular plan, so that didn't solve the issue. Then i removed them from the plans and placed them in the userregion launcher properties located in config. That hanged the deployer, shell and the model bundle in the kernel with errors that they are wating for services - here is the snippet:
osgi> [2011-02-04 02:51:57.189] service-monitor-thread-1 <KE0100W> Reference '&userRegionBundleContext' in bundle 'org.eclipse.virgo.kernel.shell' version '3.0.0.BUILD-0110204000342' is waiting for service with filter '(&(objectClass=org.osgi.framework.BundleContext)(org.eclipse.virgo.kernel.regionContext=true))'.
[2011-02-04 02:51:57.192] service-monitor-thread-1 <KE0100W> Reference '&deployer' in bundle 'org.eclipse.virgo.kernel.shell' version '3.0.0.BUILD-20110204000342' is waiting for service with filter '(objectClass=org.eclipse.virgo.kernel.deployer.core.ApplicationDeployer)'.
[2011-02-04 02:51:57.195] service-monitor-thread-1 <KE0100W> Reference '&objectNameCreator' in bundle 'org.eclipse.virgo.kernel.shell' version '3.0.0.BUILD-20110204000342' is waiting for service with filter '(objectClass=org.eclipse.virgo.kernel.model.management.RuntimeArtifactModelObjectNameCreator)'.
[2011-02-04 02:51:57.198] service-monitor-thread-1 <KE0100W> Reference '&runtimeArtifactModel' in bundle 'org.eclipse.virgo.kernel.model' version '3.0.0.BUILD-20110204000342' is waiting for service with filter '(objectClass=org.eclipse.virgo.kernel.deployer.model.RuntimeArtifactModel)'.
[2011-02-04 02:51:57.201] service-monitor-thread-1 <KE0100W> Reference '&userBundleContext' in bundle 'org.eclipse.virgo.kernel.model' version '3.0.0.BUILD-20110204000342' is waiting for service with filter '(&(objectClass=org.osgi.framework.BundleContext)(org.eclipse.virgo.kernel.regionContext=true))'.
[2011-02-04 02:51:57.207] service-monitor-thread-1 <KE0100W> Reference '&applicationDeployer' in bundle 'org.eclipse.virgo.kernel.model' version '3.0.0.BUILD-20110204000342' is waiting for service with filter '(objectClass=org.eclipse.virgo.kernel.deployer.core.ApplicationDeployer)'.
[2011-02-04 02:51:57.218] service-monitor-thread-1 <KE0100W> Reference '&userBundleContext' in bundle 'org.eclipse.virgo.kernel.deployer' version '3.0.0.BUILD-20110204000342' is waiting for service with filter '(&(objectClass=org.osgi.framework.BundleContext)(org.eclipse.virgo.kernel.regionContext=true))'.
[2011-02-04 02:51:57.222] service-monitor-thread-1 <KE0100W> Reference '®ion' in bundle 'org.eclipse.virgo.kernel.deployer' version '3.0.0.BUILD-20110204000342' is waiting for service with filter '(&(objectClass=org.eclipse.virgo.kernel.osgi.region.Region)(org.eclipse.virgo.kernel.region.name
So this also wasn't a solution. I guess the hooks will have to be modified in some way to allow the DS bundles to be visible for resolving in the user region and also expose the user region to the DS bundles as they have to process the DS components located inside the user region (such is the agent.dm
bundle, that is present both in the kernel and the user region and has two DS components). That could lead to some complications as it basically means that the DS components will become cross-region artifacts. We'll have to look deeper in this matter.
2011/2/3 Glyn Normington <gnormington@xxxxxxxxxx>
Great news. Thanks Hristo.
On 3 Feb 2011, at 13:56, Iliev, Hristo wrote:
I just tried to add the Equinox Launcher on top of your changes. The result is that we have one more issue – simple configurator refuses to install “equinox-event” bundle, but is ok with “org.eclipse.equinox.event”.
Other than this the things work ok. I tested the Web distribution and we intend to add some P2 soon.
The branch bug330776-framework-hooks now contains my latest working copy. All the tests including the smoke test pass. The framework hooks are re-implemented in terms of a directed graph ("digraph") of regions. Currently there are two permanent regions and one temporary region, which I'll briefly explain.
The permanent regions are the kernel and user region, pretty much like today except that they are defined as two nodes in the digraph connected by two arcs to do the relevant filtering. The system bundle is defined to be part of the kernel region and its packages are imported into the user region (in terms of a package filter). Later I will add the concept of importing a bundle from one region to another (the corresponding bundle filter is already there in the underlying diagraph machinery) and expose that as a user region configuration property.
The temporary region is simply a way of managing the bundle descriptions associated with a side-state during deployment dependency resolution and provisioning from the repository. Since these bundle descriptions do not correspond to real bundles and need deleting when we have finished with the side-state, I hive them off into their own region which is connected to the user region, and vice versa, by the most permissive filter "TOP" (pardon the lattice theory terminology. If someone can think of a more computerese term, please let me know.) Then rather than having to remove bundle id's from the user region when we have finished with the side state, we simply delete the temporary region. We have christened this temporary region a "coregion" (again we struggled to find a good term. Think of a co-pilot.)
Hristo: it is probably worth re-doing the merge with your branch to check the interactions again if you don't mind.
Borislav: you might consider doing the same on your branch as I may have introduced some extra ordering dependency without noticing.
Hristo/Borislav: I suggest you create a branch of your branch and then merge my branch with it for testing. That way, you will have the freedom to merge your branch into master before I am finished. Let me know if this is clear as mud.
On 31 Dec 2010, at 18:27, Glyn Normington wrote:
Even after "reverse" merging master into the bug330776-framework-hooks branch, the branch is functionally in good shape: all the tests pass and vsh now functions correctly. The branch is pushed to master.
I am part way through moving the region-specific logic from the hooks into policies associated with the regions. Ultimately, I'm aiming for a directed graph of regions connected by arcs representing package, service, and perhaps bundle, import policies. This is some way off. The hard nuts to crack are how to handle alternative paths between regions (which are equivalent to a logical OR in terms of filtering) and transitive dependencies along paths (which are equivalent to a logical AND in terms of filtering). This generality is hard to visualise in the current implicit graph with only two nodes and one arc!
The most recent git commits on the branch tell the full story for anyone who's interested:
1c30618 Merge branch 'master' into bug330776-framework-hooks
22087fd bug 330776: attach package import policy to region rather than to hook
a2fcf17 bug 330776: delete isUserRegion methods and use Region equality instead
277b221 bug 330776: move user region creation into user region factory bundle
3314e25 bug 330776: use user region factory bundle context as user region bundle context
2b5249e bug 330776: add user region factory bundle
f13f65f bug 330776: start to separate region manager and user region
91e3c58 bug 330776: delete RegionAwarePackageAdminAccessor as there is now only one PackageAdmin service
9f42a41 bug 330776: tidy up vsh change and fix test
6d7f064 bug 330776: make vsh deal primarily with user region bundles
0f4b711 bug 330776: delete inheritedFrameworkProperties
6846283 bug 330776: directive support and unit test
9e01530 bug 330776: attribute matching and unit tests
f38bebd bug 330776: factory out package import matching to be a RegionPackageImportPolicy
c401af4 bug 330776: support osgi.host capability
43ec52b bug 330776: delete old methods from RegionMembership interface
aee4936 bug 330776: add new methods to RegionMembership interface
d3671ef bug 330776: refer to bug 333193.
5b0f8dd bug 330776: complete RegionResolverHookTests, fix a bug in RegionResolverHook, and add test constants
ab5e0f4 bug 330776: add RegionResolverHookTests and improve the somewhat academic handling of system bundle in RegionResolverHook
d9e49c0 bug 330776: add RegionBundleFindHooksTests and special case system bundle (so it can find, and be found by, all bundles) in RegionBundleFindHook
16210a6 bug 330776: add RegionServiceFindHookTests
56e318f bug 330776: add RegionServiceEventHookTests and refactor RegionBundleEventHookTests
136ecf6 bug 330776: RegionServiceHookBaseTests
cefd8f3 Merge branch 'master' into bug330776-framework-hooks
aeaf89a bug 330776: RegionBundleEventHookTests
8a9ff51 bug 330776: RegionHookBaseTests and StubRegionMembership
2ae125d bug 330776: factor out abstract base class for all region framework hooks
The console output is as follows - the addition of the user region factory bundle is the biggest change apart from the basic single framework effect on bundle id numbering and visibility:
0 ACTIVE org.eclipse.osgi_3.7.0.v20101022
1 ACTIVE com.springsource.slf4j.api_1.6.1
2 ACTIVE com.springsource.slf4j.org.apache.commons.logging_1.6.1
3 ACTIVE com.springsource.org.aspectj.weaver_1.6.6.RELEASE
4 ACTIVE org.eclipse.virgo.medic_2.2.0.D-20101207150849
5 ACTIVE org.eclipse.osgi.services_3.3.0.v20101018
6 ACTIVE org.eclipse.equinox.cm_1.0.200.v20100520
7 ACTIVE org.eclipse.virgo.medic.core_2.2.0.D-20101207150849
8 ACTIVE org.eclipse.virgo.util.osgi_2.2.0.D-20101207150035
9 ACTIVE org.apache.felix.eventadmin_1.0.0
10 ACTIVE org.eclipse.virgo.util.common_2.2.0.D-20101207150035
11 ACTIVE org.eclipse.virgo.util.io_2.2.0.D-20101207150035
12 RESOLVED org.eclipse.virgo.util.jmx_2.2.0.D-20101207150035
13 ACTIVE org.eclipse.virgo.util.math_2.2.0.D-20101207150035
14 ACTIVE org.eclipse.virgo.util.parser.manifest_2.2.0.D-20101207150035
15 ACTIVE com.springsource.org.apache.commons.codec_1.3.0
16 ACTIVE com.springsource.org.apache.commons.httpclient_3.1.0
17 ACTIVE org.eclipse.virgo.repository_2.2.0.D-20101207151510
18 RESOLVED org.springframework.aop_3.0.0.RELEASE
19 RESOLVED org.springframework.asm_3.0.0.RELEASE
20 RESOLVED org.springframework.expression_3.0.0.RELEASE
21 RESOLVED org.springframework.beans_3.0.0.RELEASE
22 RESOLVED org.springframework.core_3.0.0.RELEASE
23 RESOLVED org.springframework.context_3.0.0.RELEASE
24 ACTIVE org.eclipse.virgo.kernel.core_2.2.0.BUILD-20101231170629
25 ACTIVE org.eclipse.virgo.kernel.agent.dm_2.2.0.BUILD-20101231170629
26 RESOLVED org.springframework.osgi.core_1.2.1
27 ACTIVE org.springframework.osgi.extender_1.2.1
28 RESOLVED org.springframework.osgi.io_1.2.1
29 RESOLVED com.springsource.org.aopalliance_1.0.0
30 ACTIVE org.eclipse.virgo.kernel.artifact_2.2.0.BUILD-20101231170629
31 ACTIVE org.eclipse.virgo.kernel.services_2.2.0.BUILD-20101231170629
32 ACTIVE org.eclipse.virgo.kernel.deployer_2.2.0.BUILD-20101231170629
33 ACTIVE org.eclipse.virgo.kernel.model_2.2.0.BUILD-20101231170629
34 RESOLVED org.eclipse.virgo.kernel.kerneldmfragment_2.2.0.BUILD-20101231170629
35 ACTIVE org.eclipse.virgo.kernel.shell_2.2.0.BUILD-20101231170629
36 ACTIVE org.eclipse.virgo.kernel.osgi_2.2.0.BUILD-20101231170629
37 ACTIVE org.eclipse.virgo.kernel.userregionfactory_2.2.0.BUILD-20101231170629
38 ACTIVE org.eclipse.virgo.kernel.userregion_2.2.0.BUILD-20101231170629
39 ACTIVE org.eclipse.virgo.kernel.osgicommand_2.2.0.BUILD-20101231170629
40 ACTIVE org.springframework.aop_3.0.0.RELEASE
41 ACTIVE org.springframework.asm_3.0.0.RELEASE
42 ACTIVE org.springframework.expression_3.0.0.RELEASE
43 ACTIVE org.springframework.beans_3.0.0.RELEASE
44 ACTIVE org.springframework.core_3.0.0.RELEASE
45 ACTIVE org.springframework.context_3.0.0.RELEASE
46 ACTIVE com.springsource.org.aopalliance_1.0.0
47 ACTIVE com.springsource.org.apache.commons.codec_1.3.0
48 ACTIVE com.springsource.org.apache.commons.httpclient_3.1.0
49 ACTIVE org.springframework.osgi.core_1.2.1
50 ACTIVE org.springframework.osgi.extender_1.2.1
51 ACTIVE org.springframework.osgi.io_1.2.1
52 ACTIVE org.eclipse.virgo.kernel.agent.dm_2.2.0.BUILD-20101231170629
53 ACTIVE org.eclipse.virgo.kernel.deployer.dm_2.2.0.BUILD-20101231170629
54 RESOLVED org.eclipse.virgo.kernel.dmfragment_2.2.0.BUILD-20101231170629
0 org.eclipse.osgi 3.7.0.v20101022 ACTIVE
37 org.eclipse.virgo.kernel.userregionfactory 2.2.0.BUILD-20101231170629 ACTIVE
38 org.eclipse.virgo.kernel.userregion 2.2.0.BUILD-20101231170629 ACTIVE
39 org.eclipse.virgo.kernel.osgicommand 2.2.0.BUILD-20101231170629 ACTIVE
40 org.springframework.aop 3.0.0.RELEASE ACTIVE
41 org.springframework.asm 3.0.0.RELEASE ACTIVE
42 org.springframework._expression_ 3.0.0.RELEASE ACTIVE
43 org.springframework.beans 3.0.0.RELEASE ACTIVE
44 org.springframework.core 3.0.0.RELEASE ACTIVE
45 org.springframework.context 3.0.0.RELEASE ACTIVE
46 com.springsource.org.aopalliance 1.0.0 ACTIVE
47 com.springsource.org.apache.commons.codec 1.3.0 ACTIVE
48 com.springsource.org.apache.commons.httpclient 3.1.0 ACTIVE
49 org.springframework.osgi.core 1.2.1 ACTIVE
50 S org.springframework.osgi.extender 1.2.1 ACTIVE
54 org.eclipse.virgo.kernel.dmfragment 2.2.0.BUILD-20101231170629 RESOLVED
Finally, if you've read this far, and even if you haven't, Happy New Year! ;-)
On 22 Dec 2010, at 15:31, Glyn Normington wrote:
Yesterday I got all the kernel tests passing. However, the kernel still failed to initialise when processing initialArtifacts. I discovered that the resolver hook is not driven in some situations and raised bug 333071 to request that this be implemented in Equinox.
Meanwhile I found a temporary workaround and the kernel now starts successfully (although the vsh command manifests some bugs which I will deal with in due course).
I am now going to focus on reducing the technical debt in the branch, initially by refactoring and writing unit tests. I expect to be working on the branch for the next few weeks and will merge master into the branch from time to time so I don't fall too far behind.
On 17 Dec 2010, at 16:03, Glyn Normington wrote:
That test failure appears to have been due to a configuration problem in some earlier tests. One test (DeployerLogMessageTests) fails because I have not yet implemented a service registry find hook (probably an early task for next week's sprint) to provide the necessary isolation between services in one region and bundles consuming those services in the other region.
Then when I package and run the kernel, it fails resolving the contents of the plan referred to by the initialArtifacts user region property. This is odd as the InitialArtifactDeployer uses the deployer service which works successfully in the deployer tests. More debugging required.
While we are feeling jolly, this reminds me of the old chestnut:
Real Programmers' programs never work right the first time. But if you throw them on the machine they can be patched into working order in "only a few" 30-hour debugging sessions.
I think that's what you call test driven development. ;-)
On 17 Dec 2010, at 12:30, Glyn Normington wrote:
Tom Watson suggested a workaround to enable the resolver hook to be driven for offline State resolutions and it seems to work. Details in bug 332771 and changes pushed to the branch.
Now all the kernel tests appear to pass except the last deployer test TransitiveConstraintFailureDiagnosisTests for which the JVM crashes, which usually means a heap or permgen issue. I will continue to investigate this.
On 16 Dec 2010, at 17:28, Glyn Normington wrote:
My work to rebase Virgo's region support on the framework hooks was going reasonably well until I discovered that the resolver hook is not driven for resolutions in offline State objects. I have raised bug 332771  against Equinox to cover this, but this bug won't be worked on until the New Year.
Meanwhile I have pushed my changes to the kernel branch bug330776-framework-hooks if anyone's interested. I may experiment with pruning the unwanted kernel bundles from the side state as a temporary workaround, but I don't know how well that would work as it's only an approximation to the true requirement and I haven't checked the feasibility yet.
virgo-dev mailing list
virgo-dev mailing list
virgo-dev mailing list
virgo-dev mailing list
virgo-dev mailing list