Re: [p2-dev] Re: Question about artifact mirroring

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]

Re: [p2-dev] Re: Question about artifact mirroring

From: Pascal Rapicault <Pascal_Rapicault@xxxxxxxxxx>
Date: Tue, 14 Apr 2009 20:29:34 -0400
Delivered-to: p2-dev@xxxxxxxxxxx
List-archive: <https://dev.eclipse.org/mailman/private/p2-dev>
List-help: <mailto:p2-dev-request@eclipse.org?subject=help>
List-subscribe: <https://dev.eclipse.org/mailman/listinfo/p2-dev>, <mailto:p2-dev-request@eclipse.org?subject=subscribe>
List-unsubscribe: <https://dev.eclipse.org/mailman/listinfo/p2-dev>, <mailto:p2-dev-request@eclipse.org?subject=unsubscribe>

Despite what I think, it is probably a feature that some people would be looking to use and as such we should look into integrating it post 3.5.

PaScaL

Thomas Hallgren ---04/14/2009 04:57:17 PM---I first created it to be included into my "Galileo Builder" tool (which I'm just about to release).

From:	Thomas Hallgren <thomas@xxxxxxx>
To:	P2 developer discussions <p2-dev@xxxxxxxxxxx>
Date:	04/14/2009 04:57 PM
Subject:	Re: [p2-dev] Re: Question about artifact mirroring

I first created it to be included into my "Galileo Builder" tool (which I'm just about to release). I raised the question here to see if there was an interest of getting it into P2. If so, I'll submit a patch for it.

- thomas

Jeff McAffer wrote:

Yup. the approach seems reasonable especially if it is configurable. is this something you are thinking of putting in now?

Jeff

Thomas Hallgren wrote:

Pascal Rapicault wrote:

What you are suggesting has been mentioned in the past as a way to reduce the bandwidth necessary to mirror. However I personally disagree with doing this automatically because the processing step introduces another point of variability that could be the source of problems (for example imagine that the unpacker you use when you mirror has a bug that can't be found until runtime).

In my case, it's very likely that the unpacker I use is similar to the one that will later be used by the consumers of the mirror so it's actually an advantage if I discover that some files cannot be unpacked while mirroring. That way, I can fix the problem before I make the mirror available.

If this is enabled, it has to be done through an explicit end user decision.

I agree. It should be configurable.

Also the other thing to be careful is to not create artifacts that were not in the original repo (e.g. maybe the original repo does not have the canonical artifact for a given plug-in).

My current implementation checks that. If there's no canonical artifact in the source, none is created in the mirror.

- thomas

PaScaL

Thomas Hallgren ---04/13/2009 11:03:12 AM---After some research I concluded that there was no shrink-wrapped way of doing this. So I created a

From:	Thomas Hallgren <thomas@xxxxxxx>
To:	P2 developer discussions <p2-dev@xxxxxxxxxxx>
Date:	04/13/2009 11:03 AM
Subject:	[p2-dev] Re: Question about artifact mirroring

After some research I concluded that there was no shrink-wrapped way of doing this. So I created a new class: public class CanonicalizeRequest extends MirrorRequest { private IArtifactDescriptor optimizedDescriptor; private IArtifactDescriptor canonicalDescriptor; public CanonicalizeRequest(IArtifactDescriptor optimizedDescriptor, IArtifactDescriptor canonicalDescriptor, IArtifactRepository targetRepository) { super(canonicalDescriptor.getArtifactKey(), targetRepository, null, null); this.optimizedDescriptor = optimizedDescriptor; this.canonicalDescriptor = canonicalDescriptor; setSourceRepository(targetRepository); } public void perform(IProgressMonitor monitor) { setResult(transfer(canonicalDescriptor, optimizedDescriptor, monitor)); } } The mirroring is done by first using a RawMirrorRequest on the optimized descriptor, and then using CanonicalizeRequest to unpack (source and destination both pointing to the target repo). It seems to work well. Perhaps something to consider as a more generic feature for a mirroring application? Regards, Thomas Hallgren Thomas Hallgren wrote: > Let's assume that I have a remote repository that contains two > descriptors per artifact key, one optimized (pack200) and one > canonical. I want to mirror this repository to another that later also > will be deployed as a remote repository. How do I create a mirror that > contains both the optimized and the canonical files? Copying both from > a remote location using a RawMirrorRequest doesn't seem very optimal > and using a standard MirrorRequest will transfer the optimized version > but only store the canonical. > > What I would like the mirror request to do is: > 1. Copy the optimized version over the wire. > 2. Store the optimized version verbatim. > 3. Recreate the canonical version from the optimized one and store > that too. > > Has this use-case been considered? > > Regards, > Thomas Hallgren > > _______________________________________________ p2-dev mailing list