| Re: [tycho-user] How to prepare the p2-site for an offline target platform to be used in a tycho build |
I really wonder why we need to do much more work to simple do what a tool could do for us fully automatically..i really don't get this tools IDE,RAD or command line are for us to make stuff easier and do much stuff for us without us having todo all kinds of stuffIn your scenario i suddenly have to have 3 target files that all are kind of the same1> main that really has all the external urls and plugin/feature versions2> our developer local.target that developers use for there local target created from <1>, (because eclipse is just as horrible when it comes to network errors, suddenly it is possible you just can't do our job)3> a own site that has 1 url with all features and plugins (Generated from <1>) that is then used in all the pom files. This one changes also per branch.and i need that to make a special jenkins job that needs to be run at the moment the main <1> target file changes to generate and upload to the siteof course its all possible a lot more stuff then just <useCachedMetaData>true</useCachedMetaData>in a few parent pom files..but it could be just meOn Mon, 7 Dec 2020 at 14:13, Christoph Läubrich <laeubi@xxxxxxxxxxxxxx> wrote:I don't say they do it for releases but for i-builds and that there is
nothing that prevents anyone from that.
As Mickael already noted tycho can't know if it is a "released" site a
temporary one, an composite update-site or whatever.
The problem is that there are so many (valid) use-cases that its hard to
optimize for one (for example yours) without breaking the other.
Beside that, its undesireable to offer too many options for problems
where a solution already exits (e.g. mirroring a site).
You can for example try-mirror an updatesite in your build without a
problem, if it fails your free to ignore the fact. Other might be more
scared that their update-server is gone than that this particular job
suceeds even if their infra is blowed up and want to raise an error
instead...
Am 07.12.20 um 13:48 schrieb Johan Compagner:
> are you saying that when products are released
> they sometimes make update to that with the same version?
>
> that would be really horrible. because how would the support go for that?
>
> You have a problem?
> yes.
> What version do you have?
> X.
> But X shouldn't have that problem if i check it against that version and
> it works fine..
>
> So i expect that a version that i download from a site is "static" "fixed"
>
>
>
> On Mon, 7 Dec 2020 at 13:16, Christoph Läubrich <laeubi@xxxxxxxxxxxxxx
> <mailto:laeubi@xxxxxxxxxxxxxx>> wrote:
>
> If you want to be sure nothing changes there is no other way to mirror
> all stuff to a trusted location (what could be a file location without
> any problem).
> Because for an Updatesite is pure convention that '2020127-1300' never
> changes!
>
> Anyways I think tycho can be enhanced to have some kind of "never look
> for newer versions" flag... you better create an enhancement request if
> there are strong needs for this.
>
> Am 07.12.20 um 12:11 schrieb Johan Compagner:
> >
> >
> > On Mon, 7 Dec 2020 at 11:50, Mickael Istria <mistria@xxxxxxxxxx
> <mailto:mistria@xxxxxxxxxx>
> > <mailto:mistria@xxxxxxxxxx <mailto:mistria@xxxxxxxxxx>>> wrote:
> >
> > Hi,
> >
> > On Mon, Dec 7, 2020 at 11:36 AM Johan Compagner
> > <jcompagner@xxxxxxxxxx <mailto:jcompagner@xxxxxxxxxx>
> <mailto:jcompagner@xxxxxxxxxx <mailto:jcompagner@xxxxxxxxxx>>> wrote:
> >
> > Why is it if it is already builded with that on
> jenkins for many
> > times already it can fail suddenly if one of the sites
> has a hiccup?
> >
> >
> > Rebuilding from scratch is a way to ensure build portability and
> > reproducibility in any environment. That's something that's
> usually
> > desired.
> >
> >
> > for me the target site has exact version numbers
> > if i include a git features i include "2020127-1300"
> >
> > that will not change..
> >
> > I think it is the same for all the stuff we have in our target file.
> >
> > servoy-eclipse/com.servoy.eclipse.target.target at master ·
> > Servoy/servoy-eclipse (github.com <http://github.com>)
> >
> <https://github.com/Servoy/servoy-eclipse/blob/master/launch_targets/com.servoy.eclipse.target.target
> <https://github.com/Servoy/servoy-eclipse/blob/master/launch_targets/com.servoy.eclipse.target.target>>
> >
> >
> <https://github.com/Servoy/servoy-eclipse/blob/master/launch_targets/com.servoy.eclipse.target.target
> <https://github.com/Servoy/servoy-eclipse/blob/master/launch_targets/com.servoy.eclipse.target.target>>as
>
> > far as i can see
> > if i download that target once (with that content)
> > it will never change and it is not supposed to change!
> > because that would mean that between builds suddenly have have
> new stuff
> > inside?
> > why would that be a desired implementation??
> >
> >
> > That would be horrible i am in the process of making a final
> release and
> > have first a few rc's and then the final in a couple of weeks time.
> > Then suddenly without me knowing tycho download all kind of
> different
> > features and between the latest rc and the final build i suddenly
> have
> > newer/other plugins?
> >
> >
> >
> > Why does it want to continuously check and download the
> stuff?
> >
> >
> > It checks the content of the remote repo for potential
> updates and
> > so on, but as long as you don't clean your Maven repository,
> it will
> > reuse the local artifacts that were already downloaded and not
> > redownload them.
> >
> >
> > right but it bombs out when it can't and it takes quite some time to
> > check all our stuff all the time
> > i rather then have an option right here:
> >
> > servoy-eclipse/pom.xml at master · Servoy/servoy-eclipse
> (github.com <http://github.com>)
> > <https://github.com/Servoy/servoy-eclipse/blob/master/pom.xml#L44
> <https://github.com/Servoy/servoy-eclipse/blob/master/pom.xml#L44>>
> >
> > <https://github.com/Servoy/servoy-eclipse/blob/master/pom.xml#L44
> <https://github.com/Servoy/servoy-eclipse/blob/master/pom.xml#L44>><fixedTarget>true</fixedTarget>
> >
> > and based on that tycho never checks after the initial download
> for that
> > target file (check the contents not the version) that it needs to do
> > some stuff
> >
> >
> > Why can't tycho not make a local site right from my target ?
> >
> >
> > How you deal with your target and its caching seems more to
> be your
> > specific build logic, not something really standard. Tycho
> doesn't
> > necessarily need nor download the whole content of a .target file
> > description, only necessary dependencies (according to dependency
> > resolution) will be fetched. I don't think such capability to
> make a
> > local p2 repo from a .target should be part of the default Tycho
> > workflow. However, Tycho can for sure provide some utils to
> improve
> > that.
> > And there are already such utils in the ecosystem, see for
> example
> >
> https://github.com/jbosstools/jbosstools-maven-plugins/blob/master/tycho-plugins/target-platform-utils/src/main/java/org/jboss/tools/tycho/targets/TargetToRepoMojo.java
> <https://github.com/jbosstools/jbosstools-maven-plugins/blob/master/tycho-plugins/target-platform-utils/src/main/java/org/jboss/tools/tycho/targets/TargetToRepoMojo.java>
> >
> <https://github.com/jbosstools/jbosstools-maven-plugins/blob/master/tycho-plugins/target-platform-utils/src/main/java/org/jboss/tools/tycho/targets/TargetToRepoMojo.java <https://github.com/jbosstools/jbosstools-maven-plugins/blob/master/tycho-plugins/target-platform-utils/src/main/java/org/jboss/tools/tycho/targets/TargetToRepoMojo.java>>
> > which does exactly that (take a .target and make a p2 repo
> from it).
> >
> >
> > no
> > i don't want this
> > because i use build servers and so on
> > then i constantly have to make this and upload? this to somewhere
> > And still that is then remote. i want it purely local and without me
> > knowing the filesystem om my jenkins
> > (because our our jenkins we have 3 branches of our product
> continuously
> > and all of them use a different target file)
> > So how would you do this?
> > How would you if you don't want to do anything on that server (thats
> > just in the cloud, we only have a jenkins frontend) commit
> something for
> > a specific branch that generates a local target specific for a
> branch
> > that is then used on in that branch?
> > Without ssh, anybody should be able to just check a file in and
> jenkins
> > should handle this then.
> >
> >
> > For example generate a sha hash from the contents of the
> target file
> > and cache based on that hash on local disk the full
> resolved site.
> > (or it fills up in that dir the stuff it needs or it
> encounters
> > that is not there)
> >
> >
> > Tycho already has cache in the Maven repo.
> >
> >
> > right so why does it bomb out if it can't find:
> >
> > servoy-eclipse/com.servoy.eclipse.target.target at master ·
> > Servoy/servoy-eclipse (github.com <http://github.com>)
> >
> <https://github.com/Servoy/servoy-eclipse/blob/master/launch_targets/com.servoy.eclipse.target.target#L16
> <https://github.com/Servoy/servoy-eclipse/blob/master/launch_targets/com.servoy.eclipse.target.target#L16>>
> >
> >
> <https://github.com/Servoy/servoy-eclipse/blob/master/launch_targets/com.servoy.eclipse.target.target#L16
> <https://github.com/Servoy/servoy-eclipse/blob/master/launch_targets/com.servoy.eclipse.target.target#L16>>(which
>
> > happens now and then for that)
> >
> >
> > So when a full build is done everything is in that dir
> based on
> > the sha hash of the contents of that target file
> > if that file changes it just generates a new one (i guess we
> > need to be able to clean up stuff later )
> > But as long as the sha hash doesn't change tycho doesn't
> try to
> > contact any site it always gets everything locally
> > This should speed up the builds, and we shouldn't
> suddenly have
> > breaking builds because of a network problem to one of the
> > things inside the target.
> >
> >
> > There is nothing that guarantees the content of a remote p2
> > repository doesn't change; so there is nothing that
> guarantees that
> > a .target file will resolve to the same artifacts in 2 distinct
> > executions. Caching the results and assuming the remote repos are
> > stable would break many use-cases of Tycho, where projects
> > explicitly rely on mutable remote repositories to build against
> > latest dependencies.
> >
> >
> > right but let me decide that then
> > i know it is fully stable. it never want that a target file that
> stays
> > the same has al kind of hard version it it (see above)
> > suddenly starts downloading different stuff
> > If that happens it should fail
> > i want to know that
> > because that means that suddenly our product builds on a
> different set
> > of jars that i don't expect
> > and i want to be fully in control
> > i am the person that says tycho you can download new stuff, i
> checked it
> > and i updated the version in the target file.
> >
> > But i still don't see how this could happen, because a target
> file (at
> > least ours) have hard coded versions in all the includes, it
> can't just
> > suddenly take another one..
> >
> >
> > This is kind of currently how we work with target files
> in our
> > Eclipse environments
> >
> >
> > In Eclipse IDE, you're supposed to hit "reload" from time to
> time in
> > case your .target has refs to repo with mutable content. It's the
> > same as what Tycho does (reload on every build).
> >
> >
> > nope
> > see my stuff above
> > as long as i don't change our target file to have different
> version of
> > all the features or plugins
> > the target file output is set in stone.
> > it will not change.
> >
> > Where is an example that does suddenly change?
> >
> >
> > Maybe tycho only needs to cache the
> contents.jar/artifacts.jar
> > of the sites? because the plugins/features are already
> cached
> > in the mvn repo. So what we cache based on a sha hash is very
> > little.
> >
> >
> > IIRC, it does already cache metadata from remote repos, but we'd
> > still need to query the HTTP head of repo metadata to know
> whether
> > something actually changed since last fetch and invalidate cache
> > accordingly. Maybe Tycho doesn't optimize the usage of such
> cache.
> > It would be interesting to dig in that direction.
> >
> >
> >
> > right thats what i kind of want i don't mind that this is an
> option that
> > i need to set like i said above.
> > i know my target file as long as the content doesn't change is
> set in
> > stone, the download will always be exactly the same, it does not
> need to
> > check anything
> >
> > johan
> >
> >
> > _______________________________________________
> > tycho-user mailing list
> > tycho-user@xxxxxxxxxxx <mailto:tycho-user@xxxxxxxxxxx>
> > To unsubscribe from this list, visit
> https://www.eclipse.org/mailman/listinfo/tycho-user
> <https://www.eclipse.org/mailman/listinfo/tycho-user>
> >
> _______________________________________________
> tycho-user mailing list
> tycho-user@xxxxxxxxxxx <mailto:tycho-user@xxxxxxxxxxx>
> To unsubscribe from this list, visit
> https://www.eclipse.org/mailman/listinfo/tycho-user
> <https://www.eclipse.org/mailman/listinfo/tycho-user>
>
>
>
> --
> Johan Compagner
> Servoy
>
> _______________________________________________
> tycho-user mailing list
> tycho-user@xxxxxxxxxxx
> To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/tycho-user
>
_______________________________________________
tycho-user mailing list
tycho-user@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/tycho-user
--Johan CompagnerServoy