Re: [virgo-dev] Framework hooks progress

[Glyn Normington <gnormington@xxxxxxxxxx>]

I have reverse merged master into branch bug330776-framework-hooks and everything still works so I pushed this back to the branch.

Regards,

Glyn

On 1 Feb 2011, at 22:18, Glyn Normington wrote:

The branch bug330776-framework-hooks now contains my latest working copy. All the tests including the smoke test pass. The framework hooks are re-implemented in terms of a directed graph ("digraph") of regions. Currently there are two permanent regions and one temporary region, which I'll briefly explain.

The permanent regions are the kernel and user region, pretty much like today except that they are defined as two nodes in the digraph connected by two arcs to do the relevant filtering. The system bundle is defined to be part of the kernel region and its packages are imported into the user region (in terms of a package filter). Later I will add the concept of importing a bundle from one region to another (the corresponding bundle filter is already there in the underlying diagraph machinery) and expose that as a user region configuration property.

The temporary region is simply a way of managing the bundle descriptions associated with a side-state during deployment dependency resolution and provisioning from the repository. Since these bundle descriptions do not correspond to real bundles and need deleting when we have finished with the side-state, I hive them off into their own region which is connected to the user region, and vice versa, by the most permissive filter "TOP" (pardon the lattice theory terminology. If someone can think of a more computerese term, please let me know.) Then rather than having to remove bundle id's from the user region when we have finished with the side state, we simply delete the temporary region. We have christened this temporary region a "coregion" (again we struggled to find a good term. Think of a co-pilot.)

Hristo: it is probably worth re-doing the merge with your branch to check the interactions again if you don't mind.

Borislav: you might consider doing the same on your branch as I may have introduced some extra ordering dependency without noticing.

Hristo/Borislav: I suggest you create a branch of your branch and then merge my branch with it for testing. That way, you will have the freedom to merge your branch into master before I am finished. Let me know if this is clear as mud.

Regards,

Glyn

On 31 Dec 2010, at 18:27, Glyn Normington wrote:

Even after "reverse" merging master into the bug330776-framework-hooks branch, the branch is functionally in good shape: all the tests pass and vsh now functions correctly. The branch is pushed to master.

I am part way through moving the region-specific logic from the hooks into policies associated with the regions. Ultimately, I'm aiming for a directed graph of regions connected by arcs representing package, service, and perhaps bundle, import policies. This is some way off. The hard nuts to crack are how to handle alternative paths between regions (which are equivalent to a logical OR in terms of filtering) and transitive dependencies along paths (which are equivalent to a logical AND in terms of filtering). This generality is hard to visualise in the current implicit graph with only two nodes and one arc!

The most recent git commits on the branch tell the full story for anyone who's interested:

1c30618 Merge branch 'master' into bug330776-framework-hooks

22087fd bug 330776: attach package import policy to region rather than to hook

a2fcf17 bug 330776: delete isUserRegion methods and use Region equality instead

277b221 bug 330776: move user region creation into user region factory bundle

3314e25 bug 330776: use user region factory bundle context as user region bundle context

2b5249e bug 330776: add user region factory bundle

f13f65f bug 330776: start to separate region manager and user region

91e3c58 bug 330776: delete RegionAwarePackageAdminAccessor as there is now only one PackageAdmin service

9f42a41 bug 330776: tidy up vsh change and fix test

6d7f064 bug 330776: make vsh deal primarily with user region bundles

0f4b711 bug 330776: delete inheritedFrameworkProperties

6846283 bug 330776: directive support and unit test

9e01530 bug 330776: attribute matching and unit tests

f38bebd bug 330776: factory out package import matching to be a RegionPackageImportPolicy

c401af4 bug 330776: support osgi.host capability

43ec52b bug 330776: delete old methods from RegionMembership interface

aee4936 bug 330776: add new methods to RegionMembership interface

d3671ef bug 330776: refer to bug 333193.

5b0f8dd bug 330776: complete RegionResolverHookTests, fix a bug in RegionResolverHook, and add test constants

ab5e0f4 bug 330776: add RegionResolverHookTests and improve the somewhat academic handling of system bundle in RegionResolverHook

d9e49c0 bug 330776: add RegionBundleFindHooksTests and special case system bundle (so it can find, and be found by, all bundles) in RegionBundleFindHook

16210a6 bug 330776: add RegionServiceFindHookTests

56e318f bug 330776: add RegionServiceEventHookTests and refactor RegionBundleEventHookTests

136ecf6 bug 330776: RegionServiceHookBaseTests

cefd8f3 Merge branch 'master' into bug330776-framework-hooks

aeaf89a bug 330776: RegionBundleEventHookTests

8a9ff51 bug 330776: RegionHookBaseTests and StubRegionMembership

2ae125d bug 330776: factor out abstract base class for all region framework hooks

The console output is as follows - the addition of the user region factory bundle is the biggest change apart from the basic single framework effect on bundle id numbering and visibility:

osgi> ss

Framework is launched.

id State       Bundle

0 ACTIVE      org.eclipse.osgi_3.7.0.v20101022

1 ACTIVE      com.springsource.slf4j.api_1.6.1

2 ACTIVE      com.springsource.slf4j.org.apache.commons.logging_1.6.1

3 ACTIVE      com.springsource.org.aspectj.weaver_1.6.6.RELEASE

4 ACTIVE      org.eclipse.virgo.medic_2.2.0.D-20101207150849

5 ACTIVE      org.eclipse.osgi.services_3.3.0.v20101018

6 ACTIVE      org.eclipse.equinox.cm_1.0.200.v20100520

7 ACTIVE      org.eclipse.virgo.medic.core_2.2.0.D-20101207150849

8 ACTIVE      org.eclipse.virgo.util.osgi_2.2.0.D-20101207150035

9 ACTIVE      org.apache.felix.eventadmin_1.0.0

10 ACTIVE      org.eclipse.virgo.util.common_2.2.0.D-20101207150035

11 ACTIVE      org.eclipse.virgo.util.io_2.2.0.D-20101207150035

12 RESOLVED    org.eclipse.virgo.util.jmx_2.2.0.D-20101207150035

13 ACTIVE      org.eclipse.virgo.util.math_2.2.0.D-20101207150035

14 ACTIVE      org.eclipse.virgo.util.parser.manifest_2.2.0.D-20101207150035

15 ACTIVE      com.springsource.org.apache.commons.codec_1.3.0

16 ACTIVE      com.springsource.org.apache.commons.httpclient_3.1.0

17 ACTIVE      org.eclipse.virgo.repository_2.2.0.D-20101207151510

18 RESOLVED    org.springframework.aop_3.0.0.RELEASE

19 RESOLVED    org.springframework.asm_3.0.0.RELEASE

20 RESOLVED    org.springframework.expression_3.0.0.RELEASE

21 RESOLVED    org.springframework.beans_3.0.0.RELEASE

22 RESOLVED    org.springframework.core_3.0.0.RELEASE

23 RESOLVED    org.springframework.context_3.0.0.RELEASE

24 ACTIVE      org.eclipse.virgo.kernel.core_2.2.0.BUILD-20101231170629

25 ACTIVE      org.eclipse.virgo.kernel.agent.dm_2.2.0.BUILD-20101231170629

26 RESOLVED    org.springframework.osgi.core_1.2.1

27 ACTIVE      org.springframework.osgi.extender_1.2.1

           Fragments=34

28 RESOLVED    org.springframework.osgi.io_1.2.1

29 RESOLVED    com.springsource.org.aopalliance_1.0.0

30 ACTIVE      org.eclipse.virgo.kernel.artifact_2.2.0.BUILD-20101231170629

31 ACTIVE      org.eclipse.virgo.kernel.services_2.2.0.BUILD-20101231170629

32 ACTIVE      org.eclipse.virgo.kernel.deployer_2.2.0.BUILD-20101231170629

33 ACTIVE      org.eclipse.virgo.kernel.model_2.2.0.BUILD-20101231170629

34 RESOLVED    org.eclipse.virgo.kernel.kerneldmfragment_2.2.0.BUILD-20101231170629

           Master=27

35 ACTIVE      org.eclipse.virgo.kernel.shell_2.2.0.BUILD-20101231170629

36 ACTIVE      org.eclipse.virgo.kernel.osgi_2.2.0.BUILD-20101231170629

37 ACTIVE      org.eclipse.virgo.kernel.userregionfactory_2.2.0.BUILD-20101231170629

38 ACTIVE      org.eclipse.virgo.kernel.userregion_2.2.0.BUILD-20101231170629

39 ACTIVE      org.eclipse.virgo.kernel.osgicommand_2.2.0.BUILD-20101231170629

40 ACTIVE      org.springframework.aop_3.0.0.RELEASE

41 ACTIVE      org.springframework.asm_3.0.0.RELEASE

42 ACTIVE      org.springframework.expression_3.0.0.RELEASE

43 ACTIVE      org.springframework.beans_3.0.0.RELEASE

44 ACTIVE      org.springframework.core_3.0.0.RELEASE

45 ACTIVE      org.springframework.context_3.0.0.RELEASE

46 ACTIVE      com.springsource.org.aopalliance_1.0.0

47 ACTIVE      com.springsource.org.apache.commons.codec_1.3.0

48 ACTIVE      com.springsource.org.apache.commons.httpclient_3.1.0

49 ACTIVE      org.springframework.osgi.core_1.2.1

50 ACTIVE      org.springframework.osgi.extender_1.2.1

           Fragments=54

51 ACTIVE      org.springframework.osgi.io_1.2.1

52 ACTIVE      org.eclipse.virgo.kernel.agent.dm_2.2.0.BUILD-20101231170629

53 ACTIVE      org.eclipse.virgo.kernel.deployer.dm_2.2.0.BUILD-20101231170629

54 RESOLVED    org.eclipse.virgo.kernel.dmfragment_2.2.0.BUILD-20101231170629

           Master=50

osgi> vsh bundle list

Id   Name                                           Version                          State

0    org.eclipse.osgi                               3.7.0.v20101022                 ACTIVE

37   org.eclipse.virgo.kernel.userregionfactory     2.2.0.BUILD-20101231170629      ACTIVE

38   org.eclipse.virgo.kernel.userregion            2.2.0.BUILD-20101231170629      ACTIVE

39   org.eclipse.virgo.kernel.osgicommand           2.2.0.BUILD-20101231170629      ACTIVE

40   org.springframework.aop                        3.0.0.RELEASE                   ACTIVE

41   org.springframework.asm                        3.0.0.RELEASE                   ACTIVE

42   org.springframework._expression_                 3.0.0.RELEASE                   ACTIVE

43   org.springframework.beans                      3.0.0.RELEASE                   ACTIVE

44   org.springframework.core                       3.0.0.RELEASE                   ACTIVE

45   org.springframework.context                    3.0.0.RELEASE                   ACTIVE

46   com.springsource.org.aopalliance               1.0.0                           ACTIVE

47   com.springsource.org.apache.commons.codec      1.3.0                           ACTIVE

48   com.springsource.org.apache.commons.httpclient 3.1.0                           ACTIVE

49   org.springframework.osgi.core                  1.2.1                           ACTIVE

50 S org.springframework.osgi.extender              1.2.1                           ACTIVE

51   org.springframework.osgi.io                    1.2.1                           ACTIVE

52   org.eclipse.virgo.kernel.agent.dm              2.2.0.BUILD-20101231170629      ACTIVE

53 S org.eclipse.virgo.kernel.deployer.dm           2.2.0.BUILD-20101231170629      ACTIVE

54   org.eclipse.virgo.kernel.dmfragment            2.2.0.BUILD-20101231170629    RESOLVED

osgi> 

Finally, if you've read this far, and even if you haven't, Happy New Year! ;-)

Regards,
Glyn

On 22 Dec 2010, at 15:31, Glyn Normington wrote:

Yesterday I got all the kernel tests passing. However, the kernel still failed to initialise when processing initialArtifacts. I discovered that the resolver hook is not driven in some situations and raised bug 333071 to request that this be implemented in Equinox.

Meanwhile I found a temporary workaround and the kernel now starts successfully (although the vsh command manifests some bugs which I will deal with in due course).

I am now going to focus on reducing the technical debt in the branch, initially by refactoring and writing unit tests. I expect to be working on the branch for the next few weeks and will merge master into the branch from time to time so I don't fall too far behind.

Regards,
Glyn

On 17 Dec 2010, at 16:03, Glyn Normington wrote:

That test failure appears to have been due to a configuration problem in some earlier tests. One test (DeployerLogMessageTests) fails because I have not yet implemented a service registry find hook (probably an early task for next week's sprint) to provide the necessary isolation between services in one region and bundles consuming those services in the other region.

Then when I package and run the kernel, it fails resolving the contents of the plan referred to by the initialArtifacts user region property. This is odd as the InitialArtifactDeployer uses the deployer service which works successfully in the deployer tests. More debugging required.

While we are feeling jolly, this reminds me of the old chestnut:

Real Programmers' programs never work right the first time. But if you throw them on the machine they can be patched into working order in "only a few" 30-hour debugging sessions.

I think that's what you call test driven development. ;-)

Regards,
Glyn

[1] http://www.jokebuddha.com/joke/Real_Programmers

On 17 Dec 2010, at 12:30, Glyn Normington wrote:

Tom Watson suggested a workaround to enable the resolver hook to be driven for offline State resolutions and it seems to work. Details in bug 332771 and changes pushed to the branch.

Now all the kernel tests appear to pass except the last deployer test TransitiveConstraintFailureDiagnosisTests for which the JVM crashes, which usually means a heap or permgen issue. I will continue to investigate this.

Regards,
Glyn

On 16 Dec 2010, at 17:28, Glyn Normington wrote:

My work to rebase Virgo's region support on the framework hooks was going reasonably well until I discovered that the resolver hook is not driven for resolutions in offline State objects. I have raised bug 332771 [1] against Equinox to cover this, but this bug won't be worked on until the New Year.

Meanwhile I have pushed my changes to the kernel branch bug330776-framework-hooks if anyone's interested. I may experiment with pruning the unwanted kernel bundles from the side state as a temporary workaround, but I don't know how well that would work as it's only an approximation to the true requirement and I haven't checked the feasibility yet.

Regards,

Glyn

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=332771

_______________________________________________
virgo-dev mailing list
virgo-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/virgo-dev

_______________________________________________
virgo-dev mailing list
virgo-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/virgo-dev

_______________________________________________
virgo-dev mailing list
virgo-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/virgo-dev

_______________________________________________
virgo-dev mailing list
virgo-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/virgo-dev

_______________________________________________
virgo-dev mailing list
virgo-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/virgo-dev