[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: AW: [equinox-dev] Re: [prov] Deltas for delivering updates
|
Answers embedded <pascal></pascal>
PaScaL
"Liebig, Stefan "
<Stefan.Liebig@co
mpeople.de> To
Sent by: "Equinox development mailing list"
equinox-dev-bounc <equinox-dev@xxxxxxxxxxx>
es@xxxxxxxxxxx cc
Subject
07/18/2007 08:39 AW: [equinox-dev] Re: [prov] Deltas
AM for delivering updates
Please respond to
Equinox
development
mailing list
<equinox-dev@ecli
pse.org>
First to provide some clarification for me:
The content (repositoryDescriptor) of an artifact repo is described within
the artifacts.xml.
This maps to a set of artifact descriptors where each artifact descriptor
contains an artifact
key, the artifacts download size and a list of post processing steps. The
post processing steps
seam to be currently unused currently.
<pascal> This is correct. The idea behind the post processing step was to
describe operations that had to be performed on the artifact once it had
been downloaded. The canonical example being unpack (from pack200). It
could be used for MD5 checks and things like that but I'm less sure about
this.
Also these processing steps should probably be used as negotiation between
repositories, since for example I should not download a packed200 jar if I
don't have the support to unpack it.</pascal>
Additionaly the repo descriptor contains a mapping definition to create a
url for accessing an
artifact. This url is than used to fetch the artifact.
<pascal> The mapper is an implementation detail of the simple artifact
repository. Other repository implementers may do the mapping another
way.</pascal>
How could deltas (the plural is for different delta algorithms) be
described here?
Is a delta an artifact in itself?
<pascal> Yes, however it is never explicitly requested for mirror by the
creator of a request.</pascal>
Maybe an artifact key (of a delta) contains an optional version property to
define the ´from version´?
The ´from version´ and the type of alg. are necessary to apply the delta.
The Mapper has than to consider the ´from version´ and the processing step
id for creating the url.
Would the ´from version´ better be part (data field) of a processing step?
The Mapper has than to consider also the processing steps for creating the
url.
<pascal>I like the approach where the type of delta is indicated as a post
processing step. It fits well in the model where first you check that the
target artifact repo can receive the artifact being downloaded, then
download, then apply the post processing steps, because downloading a delta
is just like downloading another artifact that could have its own
processing step, etc. Then the only thing that needs special code for is
how to find if a delta can be downloaded instead of the complete artifact
being requested.
Maybe a first thing would be to implement the support for post processing.
This will likely question how the download is currently handled (I'm not
really in love with what we have now) and may help move some
responsibilities around. For example, maybe the download should be handled
in the target repo itself.
</pascal>
Anyway, in a repository there may be stored a complete artifact, a zipped
artifact, a xdelta artifact
or a jbdiff artifact,.. of one and the same target artifact.
With that the MirrorRequest could be enhanced such that it can query the
source repo for different
variants (complete,delta,..) of the target artifact, choose the appropriate
one, download it, apply
the post processing steps and store it in the target repo.
Stefan
> -----Ursprüngliche Nachricht-----
> Von: equinox-dev-bounces@xxxxxxxxxxx
> [mailto:equinox-dev-bounces@xxxxxxxxxxx] Im Auftrag von
> Pascal Rapicault
> Gesendet: Mittwoch, 11. Juli 2007 15:50
> An: Equinox development mailing list
> Cc: equinox-dev@xxxxxxxxxxx; equinox-dev-bounces@xxxxxxxxxxx
> Betreff: [equinox-dev] Re: [prov] Deltas for delivering updates
>
> As you correctly identified the separation of responsibility
> between ECF and the provisioning code is done as follow:
> - ECF is used to do the transports (e.g http, ftp,
> bittorrent). We leverage, its API, its transport pluggability
> (and eventually its ability to restart downloads).
> - The provisioning code (DownloadManager) is responsible to
> cause the download of all the requested artifacts using the
> artifact repos. It will be responsible for picking the best
> mirror (hopefully we can reuse what is in Maya) and also
> doing the parallelization.
>
> >Can/should delta handling be part of ECF - as a kind of smart
> transport/download handling? Or is delta handling so
> important to provisioning that it should be part of provisioning?
> Given our usage of ECF, I don't think that ECF has enough
> context to be able to handle delta (it would not have the
> identification information that you are talking about).
> Therefore I think the ability to do delta depends on the
> repository in which the artifact is being mirrored to and
> from and it should be transparent for the download manager.
> Therefore with the current code I think the decision or not
> to use delta can be made in the MirrorRequest by checking the
> characteristics of the artifact repositories at hand, and the
> actual logic of getting the right delta could be left to the
> implementation of the repositories.
>
> It is also important to note that this code did not receive a
> lot of attention and anything could be changed.
>
> PaScaL
>
>
>
>
> "Liebig, Stefan "
>
> <Stefan.Liebig@co
>
> mpeople.de>
> To
> Sent by:
> <equinox-dev@xxxxxxxxxxx>
> equinox-dev-bounc
> cc
> es@xxxxxxxxxxx
>
>
> Subject
> [equinox-dev] Deltas
> for delivering
> 07/11/2007 04:49 updates
>
> AM
>
>
>
>
>
> Please respond to
>
> Equinox
>
> development
>
> mailing list
>
> <equinox-dev@ecli
>
> pse.org>
>
>
>
>
>
>
>
>
>
> I am currently investigating how the delta stuff (binary
> deltas) could fit into the ongoing provisioning work.
>
> When working with deltas there are certain constraints such as:
> - when requesting a delta for an artifact it is essential
> that the request contains an identification (version, md5,
> ..) of the artifact that is already available on the client,
> so that is possible to regenerate the new artifact from the
> old artifact and the delta.
> - the client should not rely on the fact that the server(s)
> can always respond with a delta. In some way the client needs
> to have some flexibility on handling the data (deltas,
> complete ´files´) which the server can respond.
>
> Currently provisioning uses ECF to handle the transport and download.
> Can/should delta handling be part of ECF - as a kind of smart
> transport/download handling? Or is delta handling so
> important to provisioning that it should be part of provisioning?
>
> Any thoughts?
> _______________________________________________
> equinox-dev mailing list
> equinox-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/equinox-dev
>
>
> _______________________________________________
> equinox-dev mailing list
> equinox-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/equinox-dev
>
_______________________________________________
equinox-dev mailing list
equinox-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/equinox-dev
